Real-Time Reasoning Agents in Evolving Environments
Authors: Yule Wen
Yixin Ye
Yanzhe Zhang
Diyi Yang
Hao Zhu
Venue: ICLR 2026 Poster (2026)
Keywords: Real-time reasoning
Language model agents
Parallel reasoning architecture
Primary area: foundation or frontier models, including LLMs

Abstract: Agents in the real world must make not only logical but also *timely* judgments. This requires continuous awareness of the dynamic environment: hazards emerge, opportunities arise, and other agents act, while the agent's reasoning is still unfolding. Despite advances in language model reasoning, existing approaches fail to account for this dynamic nature. We introduce *real-time reasoning* as a new problem formulation for agents in evolving environments and build **Real-time Reasoning Gym** to demonstrate it. We study two paradigms for deploying language models in agents: (1) reactive agents, which employ language models with *bounded reasoning computation for rapid responses*, and (2) planning agents, which allow *extended reasoning computation for complex problems*. Our experiments show that even state-of-the-art models struggle with making logical and timely judgments in either paradigm. To address this limitation, we propose **AgileThinker**, which simultaneously engages *both reasoning paradigms*. AgileThinker consistently outperforms agents engaging only one reasoning paradigm as the task difficulty and time pressure rise, effectively balancing reasoning depth and response latency. Our work establishes real-time reasoning as a critical testbed for developing practical agents and provides a foundation for  research in temporally constrained AI systems, highlighting a path toward real-time capable agents.

REAL-TIME REASONING AGENTS IN EVOLVING ENVIRONMENTS
Yule Wen Yixin Ye Yanzhe Zhang²
Diyi Yang
Hao Zhu
'Tsinghua University SShaggaai Jiao Tong University
Georgia Institute of Technology *Stanford University
ABSTRACT
Agents in the real world must make not only logical but also timely judgments. This requires continuous awareness of the dynamic environment: hazards emerge, opportunities arise, and other agents act, while the agent's reasoning is still unfold- ing. Despite advances in language model reasoning, existing approaches fail to count for this dynamic nature. We introduce real-time reasoning as a new problem formulation for agents in evolving environments and build Real-Time Reasoning Gym to demonstrate it. We study two paradigms for deploying language models in agents: (1) reactive agents, which employ language models with bounded rea- soning computation for rapid responses, and (2) planning agents, which allow ex- tended reasoning computation for complex problems. Our experiments show that even state-of
-fe-oof-thhttttt the-art models struggle with making logical and timely judgments in either paradigm. To address this limitation, we propose Agile Tginerr which si- multaneously engages both reasoning paradigms. AgileThinker Thinker consistently out- performs agents engaging only one reasoning paradigm as the task difficulty and time pressure risc, effectively balancing reasoning depth and response latency. Our work establishes real-time reasoning as a critical testbed for developing prac- tical agents and provides foundation for research in temporally constrained Al systems, highlighting a path toward real-time capable agents.
Moving While Agani Thinking
Partners Acting While Agent Thinking
Apples Timing While Agent Thinking
1.0
0.8
SCORE
0.6
0%
+3545
+35
0.4
0.2
0.0
lokanslstop
EASY
MEDIUM
HARD
32K
16K
8K
4K
COGNITIVE LOAD (LOW - HIGH)
TIME PRESSURE (LOW HIGH)
AgileThinker (Ours) ... Max Performance Single-Paradigm Methods
Reective (V3)
Reactive Budget Forcing)
Planning
Planning (R1 Code Polecyolicy)
Figure I: Upper: Three real-time games, Freeway, Snake, and Overcooked. Lower: Under cognitive load and time pressure, AgileThinker, engaging both reactive and planning reasoning, consistently outperforms agents that engage either of them. Scores are averaged across different games.
"Co-leading authors.

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Remember your first highway drive? White-knuckling the wheel, fixated on the car ahead. Then sud- denly your exit appears and you're re three lanes over. This captures how our brains work under pres- sure: reacting intuitively keeps us safe moment-to-momeen but deliberate planning gets us where we need to go. This example highlights a fundamental challenge: real-time reasoning (Stanovich) & West 2000; Kahneman 2011; Evans & Stanovich 2013). Under time pressures, agents must s1- multancously support timely reactions and cognitively demanding long-term planning. Remarkably, humans excel at this balancing act: shifting seamlessly between instinct and analysis as situations demand. Expert drivers eventually handle both tasks effortlessly.
However, current Large Language Model (LLM)-based agents fail to live up to this challenge. Most existing work assume that the environments only change when the agents issue an action, ignoring the dynamic nature of the world, which evolves in parallel the agent's computation. As result, despite great effort in improving agent planning with LLM reasoning, including Yao et al. (2022); Gou et al. (203)): Putta al.) (2024): Ferrag et al. (2025), under this assumption, how evaluate and improve the capability to make timely decisions still an open question.
To bridge this gap, we introduce Real-Time Reasoning Gym, the first environment for language agents to reason in dynamic environments ( Our gym consists of three real-time games: Freeway, inspired by the Atari game under the same name, Snake, an adaptation of popular game and Overcooked, a two-player version of the collaborative video game In each game, the state updates at a fixed rate regardless of whether the agent finishes its reasoning. and if no action is produced by the agent, default action is used, simulating reasoning and acting in real-time world. They challenge agents with different aspects of a dynamic environment: Freeway features dynamic hazards with moving cars, Snake involves dynamic opportunities as food appears and disappears, and Overcooked requires coordination with dynamic partners who act on their own. Real-Time Reasoning Gym is useful for studying different agent designs for real-time tasks.
To compare different design choices of real-time reasoning agents, we study two paradigms: reactive agents and planning agents ( Reactive agents ensure responsiveness by limiting computation, while planning agents are allowed to perform more extensive thinking. However, neither of them perfect: planning agents cannot easily react to changes in the environment, and reactive agents fail to make strategic decisions. We propose AgileThinkke Thinker ($3), simple yet effective method that combines the strengths of both paradigms. Unlike agents with one paradigm that must choose between speed and accuracy, AgileThinker runs two LLMs in two parallel threads: a planning thread performs extended reasoning over frozen game states, and a reactive thread outputs timely decisions within environmental update time. Specifically, the reactive thread can reference partial reasoning traces from the ongoing planning process, enabling informed real-tine time decisions without waiting for complete analysis. This also differs from prior dual-system methods (Zhang et al. 2025 Liu al.,
2024| Christakopoulou et al.) 2024). where either two systems operate independently, or one must wait for another to complete before accessing its outputs.
In this paper, we study the following research questions:
RQ1 How do environment factors affect performance agents in Real-Time Reasoning Gym?
RQ2 How to balance reaction and planning resources in AgileThinker?
RQ3 How well do the results we get with simulation in Real-Time Reasoning Gym match real- world walltime experiments?
To study these questions, we manipulate the cogmitive load and time pressure of the games, facilitat- ing systematic evaluation across both dimensions. We evaluate different design choices for reactive and planning agents, including budget foreing (Muennighoff et al., 2025) for reactive agents, code- as-a-policy (Liang et al., 2022) for planning agents. For fair comparison across agent designs, we use one model family, focusing on DeepSeek V3 and R. because they are open-source and provide transparent reasoning trajectories required by AgileThinker. To assess generality, we also exper- iment with proprietary models (App. demonstrate that ThineTh consistently outperforms balancing reactive and planning processes. This advantage is confirmed to translate to real-world scenarios through wall-clock time experiments. Ultimately, work establishes a foundation for
C,3) and observe similar performance trend. Our results
methods by effectively
2

---

Published as a conference paper at ICLR 2026
developing language agents capable of complex reasoning under time constraints, bringing Al sys- tems closer to human-level performance in dynamic environments.
2 REAL-TIME REASONING GYM
To evaluate the real-time reasoning capabilities of agents, we need environment that is (1) dy- namic: the environment state continuously updates without waiting for the agent's decision; (2) cognitively challenging: the tasks should be challenging enough so that logical reasoning needed; (3) reproducible: simulated environments to avoid non-negligible noise from hardware factors.
To achieve these three desiderata, we consider new formulation like conventional turn-based environments, where the environment steps only after the agent finishes thinking and produces an action (Figureßleftt) in Real-Time Reasoning Gym, the environment steps forward at fixed rate, even when the agent has not finished thinking. If action is produced in time, a default action is applied (Figure 2 right). This simulates the real-world situation where the environment does not delay or accelerate according to the agent's 's computation time.
problem. Un-
Games order to control the dynamics of the environment for evaluating real-time reasoning, use real-time games in our gym. We created three games to capture different challenges that a dynamic environment brings: maintaining safety when haz- ards happen, seizing transient opportunities, and coordinating with partners (Tab. I
apens1_gym.By
ote, done env.raset
abe, done ams.reset(
spent, chearve((he)
agent cott-ooeeeeenaa
apent blocked Eintatied agent..ctt()
agent teauttttanniii c DEFAILT ACTION
abs, dons, semazd on (nrcstss
dls, dons, Jowaad - cev.a (ww.alop(e)liin)
Figure 2: Agent loops in OpenAl Gym (Brockman al. 2016) and Real-Time Reasoning Gym. Constants T.E and
DEFAULT AATTION will be explained in the following 'time pressure' paragraph.
In Freeway, the agent traverses multiple lanes of bidirectional traffic, which requires constant moni- toring for oncoming cars while planning future trajectories to avoid becoming trapped mid-road. In Snake, the agent eats apples which are only available for short period of time, creating opportu- nities that must be seized quickly, while greedy food collection might lead to positions where the growing snake traps itself. In Overcooked, the agent collaborates with a scripted partner following non-stationary policy (App. $ A Efficient dish preparation requires not only planning sequence of actions, but also coordinating effectively with the partner.
Cognitive Load To systematically control how challenging the games are, make the difficulty of each game tunable through a cognitive load factor (Tab. []. In Freeway, difficulty is determined by the minimum number of steps required traverse the road, since longer paths typically introduce more detours and require deeper planning horizon. In Snake, we vary the density of obstacles, increasing route complexity and the need for look-ahead. In Overcooked, complexity is controlled by the length of an internal kitchen counter, as longer counter expands navigation complexity and stretches temporal windows for high-level goals, creating larger discrepancies long- serg-term planning and immediate execution. For each game, we design 3 levels of difficulties, easy, medium, and hard; the corresponding ranges of each level can found App. Tab. 5
Time Pressure To enable reproducible and hardware- agnossicc evaluation, we use token count as a time proxy to simulate the games, leveraging the fact that LLM decoding time scales almost linearly with output length through time-per-output-tokee (TPOT), while prefilling time becomes negligible for long sequences. This yields decoding time T Ny TPOT, where Ny is the generated token
Table 1: Different Games in Real-Time Reasoning Gym.
Game
Dynamic Aspect
Cognitive Load Factor
Evaluation Metrics
Freeway
Hazards
Min steps to finish: S
#Steps the agent takes to get to the other side #Apples the agent eats before collision #Orders completed cooperatively
Snake
Opportunities
#Obstacles: N
Overcooked
Partners
Kitchen Counter Len.: L
3

---

Published as a conference paper at ICLR 2026
Static
Dynamic
Reuctive Agent
Planning Agent
Reactive Agent
hinker (Ours)
let me
Stp Heployates.
DOWN casse future collsions.
2k tokina
DOwk
DOWN cau ruture
Valae collidion.
takles
now but...
STAY scae now
Wosdswwy
STAN drteee
Only after SThi
tokers
UPic
Dely other sogge
Klace
STAY ants
The amsor ia
After slar until cunes
Wa suell
takles
Tolunts
Agent
Observation
Action
Reference
Planning
Reactive Agent
Figure 3: Existing evaluation setups for LLM Agents often assume static setting, where the en- vironment halts while the agent completes reasoning with unlimited computation. In Real-Time Reasoning Gym, environments are dynamic, evolving regardless of agents' computation state. As illustrated in the Freeway setting, Planning Agent, which performs extended reasoning without interruption, fails to act timely while Reactive Agent, which performs reasoning strictly within en- vironment update period, lacks foresight and collides. AgileThinker combines both timely reaction and long-term planning navigate such environments effectively.
count, allowing fair comparison across deployment scenarios. We impose time pressure by letting the environment step every Nrg. or T_E in Fig. 2 tokens generated by agents. When the agent cannot produce valid action, let the environment step with DEFAULT LACIIIN (Fig. 2 In Freeway and Snake, the default action is moving in the same direction as before, and in Overcooked, the default action is to stay idle. We consider four different time pressure levels. 32k, 16k, 8k, and 4k tokens per step, from low to high pressure. As shown in Figure 3 unlike existing static evaluations, the introduction of time pressure simulates the real world dynamic environments where the world does not freeze during agent reasoning.
Evaluation Evaluation metric differs for each game. In Freeway, we evaluate the number of steps the agent takes to get to the other side, while the agent is reset to the origin every time it gets hit by car; in Snake, we count the number of apples that the agent eats before a collision; and in Overcooked, we use the number of orders that the agent and the partner completed evaluation metrics represent the capability of the agents to solve the tasks not only logically but also timely. For each game, we normalize the scores by the highest score the agent could get in that game, always have score between 0 and 1. As cognitive load and time pressure increase, expect the scores decrease. However, the scores of an agent with strong real-time reasoning capabilities should decrease slower. It is worth noting that our gym is used evaluate design choices of agent systems when the model or model family (e.g. DeepSeek-V. V3 and R1) is fixed. Therefore, cross- model comparisons may be unfair due their different tokenizers and underlying architectures.
total. These
REAL-TIME REASONING AGENTS
To address the real-time reasoning problem, we consider two solution paradigms: (1) reactive, where the agent produces new action every environment step, and (2) planning, where the agent reasons across multiple steps to generate an action plan, which is then executed until the agent resumes reasoning. In the following, discuss how to create agents following each of the two paradigms, and how these two paradigms are engaged in AgileThinkes
Reactive agents We constrain reactive agents by a token budget N;. ensuring they can respond within each environment update when N, < Nr.- We consider kinds of language models
4

---

Published as a conference paper at ICLR 2026
for reactive agents: (1) non non-thinking models that produce limited tokens for each response; and (2) thinking models that produce extended reasoning which is cut off at the token budget N₂ with budget forcing (Muennighoff et al.] 2025). In both cases, the agent produces one action per environment step, enabling immediate reaction to any change. This reactive approach is commonly used in agent systems these days.
Planning agents While reactive agents ensure timeliness, their token budget restricts the com- plexity of reasoning they can perform within a single step. In contrast, planning agents can consider their plans through multiple environment steps. We consider two formats of plans: (1) multi-step actions where thinking model is used to generate sequence of multiple actions to be executed; and (2) code plans where a thinking model is used to generate code snippet that automatically
produces actions based on observation input (Liang al. 2022 Zhang et 2025). Although (1)
is often easier to generate, (2) is more adaptive to potential changes. Both formats allow for more
deliberate, long-horizon decision-makee maninn by leveraging extended reasoning.
AgileThinker All agents in- troduced above must complete their reasoning process before taking any action. To overcome this limitation, we propose Ag- ile Teinhieer, which employs two parallel threads to achieve both timely action generation and un- interrupted deep planning. The planning thread P runs an LLMM that streams the thinking pro- cess for a multi-step action plan. Upon initialization,
Computation
Te -xx
Tx
<thine DOWN is sate but couses collttion lorgr. STAY to safe now bus..
move DOWN.
..Only UP after STAY is safe in staps. STAY now. After STAY, should...
After STAY, should wolt for buses to passepttinie STAY, UP, STAY
<tleniotat me thon. should wat for bussen Aftur that, me check fit
reason- ing process begins that contin- ues until the execution of a plan. P cannot keep up with environ- mental changes (e.g- Steps 1-2 in Fig. However, since its planning focuses on long-term objectives, many generated insights remain useful over extended horizons. In contrast, the reac- tive thread R runs a separate LLM under striet time constraints TR S Tg. making decisions based on the latest observation and the (partial) output of P (see gray and yellow arrows in Fig. 4
buses STAY!
Step
Apoot
Planning
Reactior
Reffevner
Figure 4: Two parallel threads in AgileThinker
The coordination between the two threads R and P follows a time-sharing protocol: during each environment step, P operates continuously while R. activates only in the final TR. time units. The hyperparameter TR controls the resource trade-off between the two thread. With a larger TR, the reactive thread can be more adaptive, but there will be less reasoning from the planning thread to refer to. Effectively balancing planning and reaction resources is the key to success in which will be discussed in 5
4 IS SINGLE PARADIGM ENOUGH FOR REAL-TIME REASONING?
Evaluation Setup: To investigate how cognitive load and time pressure affect the performance respectively, we conduct two series of experiments. (1) Cognitive load varies (Easy, Medium, Hard) while time pressure fixed at 8k tokens/step-
enough for non-thinking models complete their responses, yet restrictive for thinking models. The intrinsic bound N; (see Section 3 for reactive agent is set Sk. (2) Time pressure varies (NT. E {4k, 8k, 16k,32k) with medium cognitive load. Here, N; is set to 4k to ensure it remains lower than time pressure budget. We evaluate each agent 32 times (8 game seeds x LLM sampling seeds) under each setting and report
'Thinking models are the LLMs trained with reinforcement learning to incentivize reasoning before generat- ing answers (DeepSeek-AI et al. 2025). while thinkkin models are the M. that have not been specifically trained generate long reasoning.
5

---

Published as a conference paper at ICLR 2026
Freeway
Snake
Overcooked
5003E
0.0
EASY
MIDGUM
IARD
EXST
MIDHUM
HABD
PASY
MERGUM COGNITIVE LOAD
HARG
COGNITIVE LOAD
COGNITIVE LOAD
SCORE
TOKERRSTEP
TEK
TAE TIME PRESSURE
12c
TIME PRESSURE
TIME PRESSURE
AnllllllTniinker Reactive (V3)
Reactive (R1 Budget Forting)
Planning
Panniig Cade-Poliyy
Figure 5: Performance of reasoning agents in Real-Time Reasoning Gym under varying cognitive loads and time pressures. Upper: we fix time pressure at 8k tokens per step and vary cognitive load. Lower: fix cognitive load medium level and vary time pressure. Full data at App. C
the average score of these samples. Details of the environments and score calculation can be found in App. A and prompts are provided in App. $ B
Figure reports the average scores over three games, while Froures breakdown per game. The results show that reactive and planning agents fail to balance decision quality and efficiency, whereas our AgileThinker achieves robust performance under varying conditions.
Reactive agent sacrifices decision quality for efficiency. By design, the reactive agent restricts computation time less than Tg and maintains consistent performance across time pressures. However, the limit on test-time scaling also causes dramatic performance drop cognitive load increases (scores falling from 0.89 to 0.15, versus 0.88 0.50 for AgileThinker) This drop stems from its inability to consider future consequences of move carefully. As exemplified in the case study in Figure the reactive agent greedily pursues immediate rewards, falling into predictable traps while AgileThinker Thinker avoids by considering long-term survival requirements.
Planning agent optimizes for decision quality but suffers under time pressure. Plamning agent excels under relaxed time constraints but suffers from dramatic degradation when time pressure increases (scores dropping from 0.92 to 0.05, versus 0.90 to 0.58 for AgileThinker). Its fundamental flaw is obliviousness to environmental changes, executing plans based outdated observations. As illustrated in the case study, the agent is unaware that the snake has moved forward during reasoning, hence crashes into wall by following the obsolete plan. However, reactive thread in AgileThinker is informed of the latest state, thus able to adjust the output of planning thread.
The variants in 3 fail to balance speed and quality. RI with budget forcing severely degrades per- formance (0.01 < 0.39 vs. V3), as forced truncation typically results in no-ops. Even ad advanced budget-aware methods (Figure cannot prevent such drops. Meanwhile, RI with code-policy suc- ceeds only in simple algorithmic tasks like Freeway. It underperforms in complex scenarios where contextual reasoning (e.g.. TTheory-of-Mine cannot be effectively compressed into code (see App. (C.4).
5 How TO MANAGE RESOURCES BETWEEN REACTION AND PLANNING?
Effective coordination between reactive thread and planning thread P requires careful time man- agement to determine when to invoke R within each environment step. We analyze how the token budget Nrp allocated to R affects the performance of AgileThinker. We set Nr, to 8k and vary the token budget Nr. from 0.5k to 8k tokens. Figure 7 presents agent scores under different NI where we also plot the cumulative distribution function (CDF) of R's token usage across all game trajectories without constraints understand its inherent computational requirements.

---

Published as a conference paper at ICLR 2026
Reactive Agent
Planning Agent
AglleThinker (Ours)
Step
Reactive
Planning
Planning
Step
Reactive
Reactive
Pour
Stop
Reactive
Reactive
Dequuct
Planning
Planning
Stop 5
Reactive
Reactive
Nepar
Step 3 Renetive Reasoning
Step 1-3 Planning Reasoning
Step Reasoning Guidedd by Planning
61: away may
(3,5) safe.. takes 3 moves (5,2) :.. immediate collision.
First, let's vitualize the grid..
reachable before disappears
snake trapped, if any direction die... nned to think about the food at 0/225) which has short life..
5): safe.
Seems safe and reachable in moves,
(s, 2): ma. lead to trapping the snake
the previous plan).
Immediate choice head
towards (5,2))
</think>
Up (U)
Prioritize eating (2,6) with short life span Stepsi, while avoiding the risky food at
and heads to 4 [3, S)...
\boxed(R)
\boxed(U)
(Boxee(LLUUUUU)
Figure Thinking trajectories of different paradigms critical steps At step 3, Reactive Agent (V3) greedily pursues the nearest food and collides inevitably after three steps. Planning Agent (R1), still reasoning over the outdated step-I I state, defaults left. However, it correctly identifies that eating the nearest food would result in future collision, and that its lifespan is sufficient to delay consumption. Guided by the reasoning of Reactive Thread, Planning Thread in the Thinker anticipates the trap and chooses move upward toward safer food target.
We can that setting Nr. too small (e.g.. 0.5k) leads low scores, R doesn't have enough time to process strategic guidance from P and generate well-reasoned actions. Conversely, setting Nrx too large creates idle periods where has completed action generation but continues productive reasoning. Empirically, performance peaks when NT, approximates the natural token upper bound of R, as indicated by the CDF of R's token usage. This suggests that R benefits from fully utilizing its allocated time without truncation extended idling.
worth noting that the optimal time budget varies across environments and requires empirical tuning. However, AgileTThe Thinker consistently outperform single-system baselines across broad budget ranges, suggesting that rough upper bound estimations are sufficient. In {Ep we further propose dynamic adjustment mechanism for Nr at test time, which is also effective.

---

Published as a conference paper at ICLR 2026
Freeway
Snake
Overcooked
SCORE
0.6
0.6
0.4
0.4
2.0
4.0
6.0
1.0
3.0
4.0
0.5
2.0
3.0
TOKEN BUDGET (K/STEP)
TOKEN BUDGET (K/STEP)
TOKEN BUDGET (K/STEP)
(V3)
Planning
Agile
Reactive (R1 Budget Fording)
Planning (R2 Code Policy)
Reaction Thread Taken Usage CDF
Figure 7: Performance of Agile' under different reactive thread token budgets NTx The cu- mulative distribution function (CDF) shows the natural token usage of R across all game trajectories when generation is not truncated, indicating inherent computational requirements of R.
PERFORMANCE IMPROVEMENT UNDER WALL--CI.OCK TIME
To validate the practical applica- bility of our token-based simu- lation, conduct experiments using actual wall-clock time with official API of DeepSeek. Our results show token count has strong linear correlation with physical inference time. Specifically, we model this rela- tionship as T = aN + 3, where represents total runtime and N represents generated tokens. Least squares estimation on all experiment trajectories (plotted in Figure [0] yields = 0.0473 s/token, B = 334.55 with R² = 0.9986. This near-pen pear-perfect correlation validates our token-based temporal abstraction and confirms its practical relevance for real-world deployments.
Table 2: Wall-clock time performance comparison across agent systems, confirming AgileTThinkr advantages persist in real- world deployment scenarios.
Environment
Reactive (V3)
Planning (R1)
ThileTh
Freeway Snake
0.24
0.12
0.88
0.37
0.04
0.45
Overcooked
0.57
0.00
0.89
We also conduct experiments to verify that the advantage of AgileThinker remains when the game is simulated in wall-clock time. Using the derived TPOT of 0.047 s/token, evaluate agent systems with environment evolution intervals of Tg = 6 minutes, corresponding to approximately 8,000 to- kens per step. Table shows that Agile Ahinerhinker consistently outperforms both Reactive and Planning Agents in physical time. These results establish that our framework's 's benefits extend beyond theory practical applications, demonstrating applicability of our architecture for agent deployments Selligence-dmmandingg real-time tasks.
7 RELATED WORK
Evaluation Environments for LLM Agents: Existing evaluation setups for L.LM agents mostly
focus on static environments where nothing changes during episodes Yang et 2024 Zhou al.
2024b) or state pauses during LLM reasoning (Zhou et al. 22024a Shi et al. 2025 ). This unrealistic assumption risks performance drop and even safety hazards when applying LLM agents in latency- sensitive applications (Sinha et al. 2024 Zheng et al. 2025). Prior work has modeled computation delays through Delay-Aware MDPs (Chen al. (020), sticky-action schemes (Mahmood et al.) 2018), and asynchronous interactive MDPs (Travnik et al. 2018 Riemer et al. 2024), but the scope of these works is limited traditional reinforcement learning. Although some works (Liu et al.) 2024 Zhang et al. 2025) do adopt LLM agents in wall-clock time, work is the first to formalize real-time reasoning problem for LLM Agents. particular, we measure elapsed token count as a hardsare-agnostic temporal unit, enabling fair and reproducible comparison across agent systems.
Budget Control for Reasoning Models: Test-time compute improves LLM performance but in- creases inference time, with overthinking behaviors commonly observed in current reasoning mod- els (Chen et al., 2025). Budget control aims to maximize LLM performance under fixed budgets, and popular methods include early truncation (Muennighoff al. 2025), prompting (Pu al. 2025)
8

---

Published as a conference paper at ICLR 2026
These methods are effective to a certain extent, but still struggle with precise control over generated token count (Alomrani et al. 2025) and performance drops when budgets are far from adequate (Han et al. 2025). This suggests that existing techniques are inadequate for handling both loose and tight bud- get constraints within single model. The results Real-Time Reasoning Gym demonstrate that SOTA budget control methods cannot effectively balance reaction and planning, necessitating dual LLM architectures for real-time environments.
and training AAgaaral & Welleck 2025 Team al. 2025 Gemini Team 2025
Real-Time AI and Dual-Process Architectures: The trade-off between accuracy and latency is foundational problem in computer science, extensively studied ranging from real-time heuristic search to robotics (Kim et al.) 2024). Classic approaches address this by dynamically allocating compute resources, such as using cascade classifiers for efficient detection (Chen & Yuille, 2005) or frame-skipping strategies in video tracking (Luo et al. 2018 Wang et 2020). While these paradigms are well-established in traditional AI, they are being revisited in the context of Large Language Models (LLMs), where variable inference costs introduce new challenges for real-time interaction (Ginart et al. 2024). Recent works have explored real-time capabilities in embodied agents (Kim et al. 2024) and asynchronous tool usage (Ginart et al. 2024), yet unified architectural
framework for balancing reasoning depth and reaction speed remains an open question.
To address this, researchers often draw inspiration from the dual process theory, which posits that human cognition operates through two distinct components: System (fast and intuitive) and System 2 (slow and deliberate) (Evans (2033 Kahneman (01T). This cognitive model has motivated various dual-agent designs, combining fast modulessuch as finite-state machines (Zhang et al. 2022)), vision transformers (Cui al. 2025), small language models (Liu al. 2024/with powerful, computa- tionally intensive modules like LLMS with tools (Christakopoulou et al. (2024) or Large Reasoning Models (LRMs) (Zhang et al. 2025). However, most existing architectures treat these systems as separate stages (e.g., cascading) or isolated parallel processes. Agile Thinker distinctively advances this paradigm by allowing the System / (Reactive Thread) to access the real-time partial reasoning traces of System 2 (Planning Thread). This shared-context mechanism enables informed decision- making with minimal delay, effectively bridging the gap between classical real-time efficiency and modern LLM reasoning capabilities.
8 CONCLUSION
In this work, we identified and formalized real-time reasoning, fundamental challenge faced by agents in real-world deployment. We introduced Real-Time Reasoning Gym, the first gym for evaluating LLM agents in continuously evolving environments. It supports independent control of cognitive load and time pressure, using token count as a prrdware-independeer temporal measure. Our evaluation revealed critical shortcomings of existing reasoning paradigms (reactive and plan- ning reasoning). To address this gap, we proposed AgileThinker, which engages two reasoning paradigms in parallel. Experiments demonstrate that our method consistently outperforms all base- lines, with advantages growing as cognitive load increases and time constraints tighten. Future work can extend our gym to more realistic scenarios, improve coordination mechanisms between two threads, or leverage our gym train urgency- LLM agents.
LIMITATIONS
Our method investigates the real-time reasoning in LLM agents. Although our formulation gen- eral, we only conducted experiments on DeepSeek models due to two reasons: (1) open source models perform poorly in general, thus the difference between different systems is not significant enough, and (2) other commercial model providers, including OpenAI, Google and Anthropic, do not provide reasoning traces which are crucial in our evaluation. We also try our best to make sure the readers understand that we do not have any empirical evidence showing that the dual system implemented in AgileThinker is precisely modeling human dual systems. The connection and dif- ference require more rigorous evaluation.

---

Published as a conference paper at ICLR 2026
USE OF LANGUAGE MODELS
We used large language models to assist with paraphrasing and improving the readability of this paper. All research ideas, designs, experiments, analyses, and conclusions entirely our own.
REPRODUCIBILITY STATEMENT
We are committed to ensuring the reproducibility of our results. The implementation of Real-Tiine Time Reasoning Gym and AgileThinker, including all necessary code and dependencies, will be made publicly available on GitHub upon paper publication. The code can be run on any platform sup- porting DeepSeek V3 and RI models, which we use for our experiments. Detailed instructions on how to set up and reproduce our results will be provided the repository, including any additional dependencies and environment configurations.
ACKNOWLEDGMENTS
We would like to thank reviewers, members of the SALT Lab, and the Stanford NLP Group for their feedback on this project. This work was supported by grant under the NSF CAREER IIS II-22247557 and ONR 100014-24-1-2532. well the Sloan Foundation.
REFERENCES
Pranjal Aggarwal and Sean Welleck. LI: Controlling how long reasoning model thinks with reinforcement learning, 2025. URL http://///wxwv..r...oo
046.0300046697 04
Mohammad Ali Alomrani, Yingxue Zhang, Derek Li, Qianyi Sun, Soumyasundar Pal, Zhanguang Zhang, Yaochen Hu, Rohan Deepak Ajwani, Antonios Valkanas, Raika Karimi, Peng Cheng, Yunzhou Wang. Pengyi Liao. Hanrui Huang, Bin Wang. Jianye Hao, and Mark Coates. Reasoning on a budget: A survey of adaptive and controllable test-time compute in llms, 2025, URL 7/arxiv. org/ abs 22077 02076
Greg Brockman, Vicki Cheung. Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojeiech Zaremba. Openai gym, 2016.
Baiming Chen, Mengdi Xu, Liang Li, and Ding Zhao. awlay- model-based reinforcement 05440
learning for continuous control, 2020.
Xiangrong Chen and A.L. Yuille. A time-efficient cascade for real-time object detection: With applications for the visually impaired. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 05) Workshops, pp. 28-28, 2005. doi: 10.1 109/CVP. 2005.399.
Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. Do not think that much for 2+3=? the overthinking of ol-like Ilms, 2025. URL https::tp:/aatt orq/abs//4122 2412.21887
Konstantina Christakopoulou, Shibl Mourad, and Maja Matari. Agents thinking fast and slow: A talker-reasoner architecture, 2024. URL https:////aax wtwp.iiww....ab////1.01 08322
Can Cui, Pengxiang Ding, Wenxuan Song, Shuanghao Bai, Xinyang Tong, Zirui Ge, Runze Suo, Wanqi Zhou, Yang Liu, Bofang Jia, Han Zhao, Siteng Huang, and Donglin Wang. Openhelix: A short survey, empirical analysis, and open-source dual-system vla model for robotic manipulation. 2025. URL /RRLihttss///rree http:///aaivvvvvia wttp:ii.wr//bbb///2222
abs
03912
DeepSeek-AL, Daya Guo, Dejian Yang. Haowei Zhang. Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu. Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan,
10

---

Published as a conference paper at ICLR 2026
Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang. Han Bao, Hanwei Xu, Haocheng Wang. Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. I.. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang, Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye. Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun, Tian Pei, Tianyu Sun, Wang. Wangding Zeng, Wanjia Zhao, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang. W. L.. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nic, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xi- aosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang. Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi. Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang. Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha. Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. Deepseck-rl: Incentivizing reasoning capability in Ilms via reinforce- ment learning. 2025.
/ttp//aavvvv/..///..//.../...ava org/oaggga....... wtwg/ab//////ooo./////ggg///aaer 2501. 12948
Jonathan St. B. T. Evans and Keith E. Stanovich. -rrc-sss theories of higher cognition: Advancing the debate. Perspectives on Psychological Science, 8(3):223-241, 2013. doi: 10.1 11177/1775591612266000 PMID: 26172965.
Jonathan St BT Evans. Dual-process theories of reasoning: Contemporary issues and developmental applications. Developmental Review, 33(2): 140- 2013.
Mohamed Amine Ferrag, Norbert Tihanyi, and Merouane Debbah. From Ilm reasoning to au- tonomous ai agents: A comprehensive review. arXiv preprint arXiv:2504.19672 2025.
Gemini Team. Gemini 2.5: A multimodal ai model. https::/stoorgee.poog gow.:aaap...ogeepp......... httpllaapiccom/o/oeeeoo.mmm. deepmi no- a /emi ni ni / ni ___2____ 5 2_tep_orrr.pp.t.. January port, Google Deep.ind.
January 2025. Technical Re-
Antonio A. Ginart, Naveen Kodali, Jason Lee, Caiming Xiong, Silvio Savarese, and John Em- mons. Asynchronous tool usage for real-time agents, 2024. URL httts::/arxiv..r
2is/2411.22620
Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang. Minlie Huang, Nan Duan, and Weizhu Chen. Tora: A tool-integrated reasoning agent for mathematical problem solving. arXiv preprint arXiv:2309.
2023.
Tingxu Han, Zhenting Wang, Chunrong Fang, Shiyu Zhao, Shiqing Ma, and Zhenyu Chen. Token- budget-aware 11m reasoning, 2025. URI. httphhttt///iiiiiieee
Daniel Kahneman. Thinking, fast and slow. Farrar, Straus and Giroux, 2011.
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Ben- jamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. Openvla: An open-source vision-language-aciin model, 2024. URL httltttts///axxi.rrrr abs/2406. 09246
Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as policies: Language model programs for embodied control. In arXiv preprint arXiv:2209.07753. 2022.
11

---

Published as a conference paper at ICLR 2026
Jijia Liu, Chao Yu, Jiaxuan Gao. Yuging Xie, Qingmin Liao, Yi Wu, and Yu Wang. Lim- powered hierarchical language agent for real-time human-ai coordination, 2024. URL https:
aaxiiv.rr//bba.. /tt.t/////i..... 15224
Hao Luo, Wenxuan Xie, Xinggang Wang, and Wenjun Zeng. Detect or track: Towards cost-effective video object detection//racking, 2018. hwt./httts//aata www./orggab////ee. 811 05340
A. Rupam Mahmood, Dmytro Korenkevych, Gautham Vasan, William Ma, and James Bergstra. Benchmarking reinforcement learning algorithms real-world robots, 2018. URL https:
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannanch Hajishirzi. Luke Zettlemoyer, Percy Liang, Emmanuel Cands, and Tatsunori Hashimoto. sl: Simple test-time scaling, 2025. URL
aaaaab//1111. 19393
Xiao Pu, Michael Saxon, Wenyue Hua, and William Yang Wang. Thoughtterminator: Bench- marking, calibrating, and mitigating overthinking in reasoning models, 2025. URL https:
//arxiv.org www./r//s//.//.oooeooo./e.eeeeee 2250/ 2t1//1100111111111 13367
Pranav Putta, Edmund Mills, Naman Garg, Sumeet Motwani, Chelsea Finn, Divyansh Garg, and Rafael Rafailov. Agent q: Advanced reasoning and learning for autonomous ai agents. arXiv preprint arXiv:2408.00 2400:27998,0 2024.
Matthew Riemer, Gopeshh Subbaraj. Glen Berseth, and Irina Rish. Enabling realtime reinfrrmmmnt learning at scale with staggered asynchronous inference, 2024. URL httthtts///axxi/// orr//
abs/ 0los24111 221224112222 1.e.14435 14355
Jiajun Shi, Jian Yang, Jiaheng Liu, Xingyuan Bu, Jiangjie Chen, Junting Zhou, Kaijing Ma, Zhou- futu Wen, Bingli Wang. Yancheng He, Liang Song, Hualei Zhu, Shilong Li, Xingjian Wang. Wei Zhang, Ruibin Yuan, Yifan Yao, Wenjun Yang. Yunli Wang. Siyuan Fang, Siyu Yuan, Qianyu He, Xiangru Tang, Yingshui Tan, Wangchunshu Zhou, Zhaoxiang Zhang. Zhoujun Li, Wenhao Huang. and Ge Zhang. Korgym: A dynamic game platform for Ilm reasoning evaluation, 2025. URL http:: //axxvv org/ aww.a//....22. 14552
Rohan Sinha, Amine Elhafsi, Christopher Agia, Matthew Foutter, Edward Schmerling, and Marco Pavone. Real-time anomaly detection and reactive planning with large language models, 2024.
Keith E. Stanovich and Richard F. West. Individual differences in reasoning: Implications for the rationality debate? Behavioral and Brain Sciences, 2365)664,665, 2000. doi: 10.1017/ 50140525X00000343..
Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, Chuning Tang, Congcong Wang, Dehao Zhang, Enming Yuan, Enzhe Lu, Fengxiang Tang, Flood Sung, Guangda Wei, Guokun Lai, Haiqing Guo, Han Zhu, Hao Ding, Hao Hu, Hao Yang, Hao Zhang. Haotian Yao, Haotian Zhao, Haoyu Lu, Haoze Li, Haozhen Yu, Hongcheng Gao, Huabin Zheng. Huan Yuan, Jia Chen, Jianhang Guo, Jianlin Su, Jianzhou Wang. Jie Zhao, Jin Zhang, Jingyuan Liu, Junjie Yan, Junyan Wu, Lidong Shi, Ling Ye, Longhui Yu, Mengnan Dong, Neo Zhang, Ningchen Ma, Qiwei Pan, Qucheng Gong, Shaowei Liu, Shengling Ma. Shupeng Wei, Sihan Cao, Siying Huang. Tao Jiang, Weihao Gao, Weimin Xiong, Weiran He. Weixiao Huang, Weixin Xu, Wenhao Wu, Wenyang He, Xianghui Wei, Xianqing Jia, Xingzhe Wu, Xinran Xu, Xinxing Zu, Xinyu Zhou, Xuchai Pan, Y. Charles, Yang Li, Yangyang Hu, Yangyang Liu, Yanru Chen, Yejie Wang, Yibo Liu, Yidao Qin, Yifeng Liu, Ying Yang. Yiping Bao, Yulun Du, Yuxin Wu, Yuzhi Wang, Zaida Zhou, Zhaoji Wang, Zhaowei Li. Zhen Zhu, Zheng Zhang, Zhexu Wang, Zhilin Yang, Zhiqi Huang, Zihao Huang, Ziyao Xu, Zonghan Yang, and Zongyu Lin. Kimi k1.5: Scaling reinforcement learning with Ilms, 2025. URL // arwii. org/ abs 2tt.:/501.11555
Jaden B. Travnik, Kory W. Mathewson, Richard S. Sutton, and Patrick M. Pilarski. Reactive rein- forcement learning in asynchronous environments. Frontiers in Robotics and Al, 5, June 2018. ISSN 2296-9144. doi: 10.3389/frobt.2188.00777 URL http:/ / dx. doi org/10, 3389/
Frobt. 2018 00079
12

---

Published as a conference paper at ICLR 2026
Zhongdao Wang, Liang Zheng, Yixuan Liu, Yali Li, and Shengjin Wang. Towards real-time multi- object tracking. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (eds.), Computer Vision ECCV 2020, pp. 107-122, Cham, 2020. Springer International Publishing. ISBN 978-3-030-58621-8.
John Yang, Carlos Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofrr Press. Swe-agent: Agent-computer interfaces enable automated software engineering, 2024. URL
wttp://o.aabbba...aaa 15793
Shunyu Yao, Jeffiey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models, 2022.
Shao Zhang, Xihuai Wang, Wenhao Zhang, Chaoran Li, Junru Song, Tingyu Li, Lin Qiu, Xuezhi Cao, Xunliang Cai, Wen Yao, Weinan Zhang, Xinbing Wang, and Ying Wen. Leveraging dual process theory in language agent framework for real-time simultaneous human-ai collaboration, 2025. URL CtRpsttps: wwwaa.iviiv... /bt.//b..//212222222
Yangqing Zheng, Shunqi Mao. Dingxin Zhang, and Weidong Cai. Lim-enhanced rapid-reflex async- reflect embodied agent for real-time decision-making dynamically changing environments. arXiv preprint arXiv:2506.0722) 2025.
Qinhong Zhou, Sunli Chen, Yisong Wang, Haozhe Xu, Weihua Du, Hongxin Zhang, Yilun Du, Joshua B. Tenenbaum, and Chuang Gan. Hazard challenge: Embodied decision making in dy- namically changing environments, 2024a. htt.:ttpp//aaaxvv.oro///////22.2
Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. Webarena: A realistic web environment for building autonomous agents, 2024b. Utthtts:: www.vvorggaaae /bs/
2307. 13854
A ENVIRONMENT DETAILS
Freeway:
The player navigates across parallel highways with moving cars. each step the player can move to adjacent lane or stay in place. Cars move forward at constant speeds and new cars may spawn on either side of the road. If the player is hit by a car, will be reset to the starting position. The game terminates if the player crosses the road or if the step limit M = 100 is reached. The reward for a trajectory T computed as:
R(T) = M - ||
The player controls a snake in a 2D rectangular grid with surrounding walls and internal obstacles. At each step, the snake can move step left, right, up down. If the snake head collides with an obstacle or its body segment, it dies. Foods spawn continuously in the map and disappears after fixed number of steps; eating food increases the snake's length by unit. The game terminates if the snake dies or the number of steps exceeds threshold M 100. The reward is calculated as:
Snake:
R(7) Number of eaten food - i[Des in M steps]
Overcooked: A fully observable two-player cooperative game where players must collect onions, cook them in pot and serve the cooked soup for rewards. At each step, the players can move in 4 directions or use interact action to trigger some events, such as picking or placing an item depending on the game state. Since we focus on single-agent settings, we model the second player as part of the changing environment, controlled by a manually written script for simplicity and consistency. This agent randomly chooses one policy to follow: deliver an onion into an arbitrary por or a kitchen counter. The game runs for M 100 steps, and rewards assigned for accomplishment of special events listed in Table 3 The game is implemented based on the repository ht /ighhtb.... com/ Huma HumanCompat www.mpppatbblet/eeee..eee
13

---

Published as a conference paper at ICLR 2026
Event
Reward
Picking up dish from the dispenser Picking up cooked soup from the pot Serving the soup
20
Table 3: Rewards for different events in the Overcooked environment.
The reward is then normalized to [0, 1] to get the final game score. Specifically, let Raiin and Rmax be the minimum and maximum rewards observed all trajectories. The score S is computed as: R- Empirically, the Rax and Rain of each environment are listed in Table
Racs - Ruis
Environment
Rmin
Rymax
Freeway
0
89
Snake
-1
15
Overcooked
0
56
Table 4: Minimum and maximum rewards for each environment.
Table 5: Game difficulty settings, showing ranges for easy, medium, and hard levels.
Game
Dynamic Aspect
Cognitive Load Factor
Easy
Medium
Hard
Freeway
Hazards
Min steps to finish: S
$ < 12
13 < $ < 16
17 < $ 21
Snake
Opportunities
# #Ohstacles: N
N =
2 N 55
6 N 3
Overcooked
Partners
Kitchen Counter Len.;
L=
B
PROMPT
Prompt for the Planning Agent
Now a player is playing a multi-turn game, and suppose current turn (t1}. Given the initial position (0, yt, 2D grid (vertical axis y 0,1,..., 9), determine the minimal number H- of turns H and sequence of actions (at,++ 1 to reach (0, 9), avoiding collisions with cars on freeways 8.
1. Game Dynamics:
Player update: Vc+1 Vt + Aye, where
+1
if a. U
Agr.
if a; D
1/41 E [0,9]
if a; = S
Car update rules:
For car k on freeway i, suppose its head is at h, tail is at J at turn t,, and speed is s. Then at turn 7' f1. the car span becomes:
Left-moving: Span(t1) [h, 7] - Span(7) -
Right-moving: Span(t1) [r. Span(T) -
Collision occurs at turn T only if 0 Span(T) for any freeway yr-
14

---

Published as a conference paper at ICLR 2026
Note that if you decide to move to vr+1 * yr turn T, you will NOT be con- sidered to be on yr-1 at turn T. thus will NOT be collided by cars on yr+1 if 0 Span(T) but & Span(T + 1).
2. Task (Turn t
Find sequence of actions {ar, + 18-1 which minimizes H such that yr, + 1-1 = 9.
Answer Format:
Turn f1: ar Turn 1 + 1: 011-1
3. Current State (Turn 1
Current Turn: to 10
Player Position: (0,6)
Car State:
Freeway & Cars (head h, tail т, direction d, speed s)
(48, 37, right, 12). (0, - 11, right, 12)
(48, right, 48)
(48, 37, right, 12), (0, - 11, right, 12)
15

---

Published as a conference paper at ICLR 2026
Prompt for the Reactive Agent
Prompt:
You are a player in freeway game, starting at (0, S/t on 2D grid (vertical axis = 0, .......... ........) 9). Your goal is to reach (0, 9) while avoiding collisions with cars on freeways y = 1, ,
1. Game Dynamics:
Player update:
y++1 y: + Aye. where
if at
A
-1 if ae D
U++1 [0.9)
if ar S
Car update rules:
For car & on freeway i, suppose its head is at h, tail is at 7 at turn to. and speed is Then at turn '' fo, the span becomes:
Left-moving: Span(to) [h, 7] + ST)n(7) = 1h - a(T to), T a(T - tu))
Right-moving: Span(fo) = [7, h]] + Span(T) = [T + s(T to), + s(T T tn)]
Collision occurs at turn T only if 0 STan(T") for car on freeway yr.
Note that if you decide to move to YT+1 * yr at turn T. you will NOT considered to be on yr+1 at turn T, thus will NOT be collided by cars on yr if E Span(T) but 0 Span(T + 1).
2. Guidance from a Previous Thinking Model (Turn t1 S fa):
Sometimes, you have access to past output from a thinking model, computed at turn & based on then-current observations. This guidance may longer perfectly match the current situation but still be valuable decision -eriii..mmaking.... You can use this plan as strategic reference, not mandatory instruction. Consider how much of the original strategy still valid under the current dynamics.
3. Task (Turn lo):
Choose action at, (U, S, for the current turn, with the following considcrations:
Collision Avoidance: Ensure the action avoids both immediate and near-future collisions.
Strategic Consistency (Optional): Refer to thinking model's prior strategy. the cur- rent environment still aligns with its assumptions, you may choose to continue along same strategic direction. If not, adapt as needed.
Answer Format:
Gto
4. Current State (Turn to):
Current Turn:
Player Position: (0, 6)
Car State:
16

---

Published as a conference paper at ICLR 2026
Freeway
Cars (head tail direction speed
(48, 37, right, 12), (0, - right, 12) right, 48)
48. left, 24)
4
( 18. - 59, right, 12). (- 12, 23, nigh, 12), (21, right 12)
24, right 4)
24, right, 4), (52. 41, nght, 4), (16,,
(48, 37, right, 12), (0, -1 11, right. 12)
(48, 37, night, 12), (0, -11 nghh, 12)
Guidance from Previous Thinking Model: Turn 10
Previous thinking process:
First. I need the player from (0, 6) (U, in few possible, avoiding collisions with cars the Ireeways to = The current turn is b 10.
up (U), down stay changing y 11, respectively, and 9 inclusive.
The player
stay between
Cars move specific frocways, and noed avoid being freeway collision happens only at turn I', the player on freeway and for that freeway, there's a car where in its span
position that The
Importantly when move new freeway, I'm not considered on that freeway until the next turn, So I move from to g at tum turn
on y, and only occupy y + turn That means don't collide with on 1 at turn only turn + if I'm still there.
Now. the goal to reach = 9. Since we start y = 6, and y = 9 is three levels up, but have to go through = and and avoid
there.
The minimal number turns might be more because might need wait or something avoid collisions.
H the number of tums, from turn & H if start 10. and take if turns, I end turn 10 +
need 1/o H
Since I start at = 6, and need to get to = 9, which is ++3, so minimum Hf would be if can go up three times without colliding. but need check that's possible given the positions.
But the cars are moving. I need to see when they are at a U. First, let's understand the car movements.
spans given t1=10.
For cach car, need to find when is its span for future tums. span changes lincarly
based speed direction.
Let me denote the time elagsed from 11=10 as At T' 10.
T, span
left-moving: [h s" At, r At] since moves left, decreasing x.
left-moving. Spun([]) (hh-s"(I<t), s((-0))
Similarly for right- moving: Spun(T) = |r (T(l))) I1, s(()))
And collision 0 Span(T) for the froeway
Now, since the player and cars are freeway. I need for each freeway k. and For each car on that treeway. find the span.
Then, to avoid collisson, when I'm on freeway l tur T. none of the cars on should have in span
Now, path to yw9.
Since y=9 the and no cars above, but I need pass through y=7 and ym8.
y=9 cars
are on y=l to
y=9, I'm
But I need there.
Start y=6 tm10.
need to to y=7. then y=8, then y=9.
might stay or move
avoid cars, moving down might help since go up eventually.
Since the grid is small, from 9, and start at 6, going it's only steps H min is 3, but if not possible, need more turns by staying or moving down and up.
17

---

Published as a conference paper at ICLR 2026
Prompt for Code-As-Policy
Prompt:
Now player is playing multi-tumn game, and suppose current turn is {tr). Given the initial position (0, (t,) on a 2D grid (vertical axis y 0, l,.... 9). You are tasked with implementing Python function that determines the optimal next action to avoid collisions with moving cars and progress towards the goal position (0, in minimal
1. Game Dynamics:
Player update:
W++1 = Ye + Aye, where
+1 if at U
Aye
a D U++1 [0.9)
d:
Car update rules:
For car & on freeway suppose head is tail is 7 at turn 21, and speed Then turn f1, the car span becomes:
LLet-moving: Span(t1)
[h, r] + STan(T) Ih a(T -1), a(T f1) Right-moving: Span(t1) = [r, A] Span(T) = (r + -(' 41), s(T - t:)]
Collision occurs at turn T only 0 Span(T") for any car on freeway yr.
Note that if you decide to move to 97+1 * yr at turn T. you will NOT be considered to be yr+1 at turn T, thus will NOT be collided by 0 Span(T + 1).
yr41 if 0 Span(T") but
2. Task
You need to determine the best next action for the player by generating an executable Python function next_action fjson_state with the input json_state representing the current game state as ISON object. The function should analyze the game state and return the next action, represented as single character string:
moving up (to y + 1)
'D" for moving down (to 1)
'S' for staying in the current position
Notice that the code will be executed in loop, so should return the next action each time it is called with the current game state, which will change after each action.
Input Format
json state
'player 'player_states's states';
current_y ppositioon int: 0-9, is the goal
position
'car_states::
list of tuples
(lane, head, direction, speed, span), lane: 1-8 (freeway number)
head: int, position of the car's head
direction: 'left' or 'right', tail head + span if left, head span if right
speed: int, speed of the car span: int, span of the car, defined the absolute di difference between head tail
1.
turn': porrent_turmnunbte $ int: current turn
18

---

Published as a conference paper at ICLR 2026
Output Format
Generate two clearly labeled parts:
Part I: Summary
One-sentence intent describing your strategy for the next actions
Part 2: Python Function
def next_action (json_state) -> str:
**"
Returns one of the actions: '0', 'o', or rs' based on the current game state.
Args:
json_statee state: The current game state as JSON object
Returns:
str: Single character representing the next action ('0,, r9 )
Your logic here return action
Example Output
Part I: Summary
Wait for the cars in lane and to pass, then move up to avoid collisions.
Part 2: Python Function
def hext_action (json_state) -> str:
# Implementation... return ''' default action if no immodiate threat
Current State
"player_states" states":
"car_states"
(1, -39, 'right', 3, 11), (2, 48, 'right', 12, 11),
(2, -36, 'right', 12, 11), (2, 0, 'right', 12, 11),
(3, 48, rright', 12, 11), (3, -36, 'right' 12, 11),
12), (3, 0, rright', 12, 11), (4, 48, 'right", 12, 11),
8
(4, -36, "right', 12, 11), (4, 0, 'right', 12, 11),
(5, -18, 'right', 6, 11), [6, -18, 'zight', 6, 11),
(7, -16, 'right', 4, 11), (8, -16, 'zight', 4, 11)
1,
"turn":
19

---

Published as a conference paper at ICLR 2026
C ADDITIONAL EXPERIMENT RESULTS
C.I MAIN RESULTS
Table 6: Complete agent performance across various cognitive load levels (Easy, Medium, Hard) with time pressure fixed at 8k tokens/step.
Freeway
Cognitive Load
Code-as-Policy
Reactive (R1)
Reactive (V3)
Planning (R1)
AgileThinker
Easy
0.5393
0.2022
0.9775
0.3371
0.9551
Medium
0.6966
0.0000
0.3258
0.2247
0.8427
Hard
0.5281
0.0000
0.0562
0.1011
0.5056
Snake
Cognitive Load
Code-as-Policy
Reactive (R1)
Reactive (V3)
Planning (R1)
AgileThinker
Easy
0.1719
0.1719
0.7694
0.0588
0.6931
Medium
0.1797
0.2188
0.4900
0.0256
0.5413
Hard
0,0625
0.1406
0.2950
0.0137
0.3906
Overcooked
Cognitive Load
Code-as-Policy
Reactive (R1)
Reactive (V3)
Planning (R1)
AgileThinker
Easy
0.4621
0.8193
0.9188
0.5379
1.0000
Medium
0.3724
0.6607
0.3664
0.0871
0,9152
Hard
0.3661
0.2054
0.0877
0.0000
0.5982
Table 7: Complete agent performance across time pressure levels (4k to 32k tokens/step) with cog- nitive load fixed at Medium.
Freeway
Tokens/Turn
Code-as-Policy
Reactive (R1)
Reactive (V3)
Planning (R1)
AgileThhiner
32k
0.9438
0.0000
0.2911
0.9621
0.9431
16k
0.9551
0.0000
0.2911
0.9045
0.9347
8k
0.6966
0.0000
0.2911
0.2261
0.8469
4k
0.3483
0.0000
0.2911
0.1194
0.6166
Snake
Tokens/Turn
Code-as-Policy
Reactive (R1)
Reactive (V3)
Planning (R1)
AgileThinker
32k
0.2109
0.0238
0.4844
0.9629
0.8281
16k
0.2344
0.0238
0.4844
0.4043
0.7813
8k
0.1797
0.0238
0.4844
0.0254
0.5410
4k
0.0156
0.0238
0.4844
0.0176
0.4238
Overcooked
Tokens/Turn
Code-as-Policy
Reactive (RI)
Reactive (V3)
Planning (R1)
AgileThinker
32k
0.5804
0.0000
0.3800
0.8371
0.9129
16k
0.4621
0.0000
0.3800
1.0000
0.9375
8k
0.3661
0.0000
0.3800
0.0871
0,9152
4k
0.3724
0.0000
0.3800
0.0246
0.7087
20

---

Published as a conference paper at ICLR 2026
C.2 SIGNIFICANCE TEST
We investigate the significance of advantage of AgileThinker single-paradigm agents. We hy- pothesize that: (1) AgileThinker's advantage over reactive agents (V3) becomes more significant as cognitive load increases, and (2) its advantage over planning agents (R1) becomes more significant as time pressure increases.
To validate these hypotheses, we perform experiments across cognitive load levels (Easy, Medium, Hard) and three 3 pressures (High: 32k tokens/step. Medium: 8k tokens/step. Low: 4k tokens/step). Below, we formally describe the validation procedure (1) and (2) can tested similarly.
For each fixed cognitive load and time pressure condition, we analyze the paired score differences between AgileThinker and the reactive agent. Let jaa denote the mean score difference between the paired observations (AgileThinker minus reactive agent). We pair observations that share the same environmental configuration seed, with each score calculated as the average across experimental runs within that configuration seed. We formulate the hypotheses follows:
Null Hypothesis: Ho pa
Alternative Hypothesis H, pa > 0
The test statistic for the paired t-test is calculated as:
d
sal Vn
where d is the mean score difference, 8, is the standard deviation of the differences, and n is number of environment configuration seeds. We estimate the p-value based the t-statistic with degrees of freedom.
Figure presents the p-values across different conditions, with statistical significance assessed at = 0.05. The results show that AgileThinker's ad advantage generally becomes statistically signifi- cant as cognitive load and time pressure increase.
Freeway
Snake
Overcooked
1.0
Ours Reactive
32k
0.27
0.00
0.00
32k
0.00
0.00
0.00
32k
0.16
0.00
0.00
8k-
8k
8k-
4k-
0.97
0.07
0.20
4k
0.84
0.91
0.33
0.63
0.09
0.00
Easy
Easy
Easy
Medium
Ours Planning
32k
32k
8k-
0.00
0.00
0.00
8k-
0.00
0.00
0.00
0.00
0.00
0.00
4k
0.00
0.01
0.05
4k
0.00
0.00
0.00
4k-
0.00
0.00
0.00
0.0
Easy
Modium
Hard
Easy
Medium
Hard
Easy
Medium
Hard
Figure 8: Statistical significance of AgileThinker's advantage over single-paradigm agents. Up- per: Advantage over reactive agent (V3). Lower: Advantage over planning agent (R1). Numbers represent p-values under varying cognitive loads and time pressures, with red indicating statistical significance (p < 0.05). The advantage of AgileThinker generally increases with both cognitive load and time pressure.
21

---

Published as a conference paper at ICLR 2026
Table DeepScek-V3.. V3.22 based agents' performance across different time pressure levels (4k 32k tokens/step) with cognitive load fixed at High.
Freeway
Tokens/Turn
Reactive (thinking off)
Planning (thinking on)
AgileTThinke
32k
0.4659
0.8539
0.8889
16k
0,4659
0,4649
0.8328
8k
0,4659
0.1025
0.6352
4k
0.4659
0.0000
0.4466
Snake
Tokens/Turn
Reactive (thinking off)
Planning (thinking on)
AgileThhinkr
32k
0,2556
0.7381
0.5700
16k
0,2556
0.3906
0.5350
8k
0.2556
0.0550
0.3456
4k
0,2556
0.0075
0.2738
Overcooked
Tokens/Turn
Reactive (thinking off)
Planning (thinking on)
AgileThinker
32k
0.5246
0.8371
0.8550
16k
0,5246
0,4554
0,8438
8k
0.5246
0.0134
0.6563
4k
0.5246
0.0068
0.4486
Table 9: DeepSeek-V3.2 based agents' performance with time pressure of 350 seconds per step and high cognitive load (Hard difficulty), confirming AgileThinker's advantage persists in wall-time constraints.
Environment
Reactive (thinking off)
Planning (thinking on)
AgileThinker
Freeway Snake
0.3665
0.0
0.7022
0.3050
0.0863
0.6013
Overcooked
0.5246
0.0
0.6675
C.3 RESULTS OF OTHER MODELS
Since AgileThinker relies on transparent reasoning trajectories. which are only available in open- source models, our primary experiments are conducted using the state-of-the-art open-source DeepSeek models. Besides using DeepSeek- V3 and DeepSeek-RI as the reactive and planning model, we also conduct experiments with DeepSeek- V3.2, where non-thinking mode is used as reactive and thinking mode used as planning. The results are shown Tab, and Tab. 9
To assess the generalizability of our approach, we also evaluate reactive and planning agents using other models, such as Gemini-2.5- reactive agents, we disable extended thinking, while for planning agents, thinking is enabled.
which features an intrinsic budget control function. For
However, we cannot directly implement AgileThinker with Gemini due to the lack access to its reasoning traces. Instead, we approximate the system using reduced design: the reactive agent references the final output the planning agent after it completes reasoning, rather than accessing partial reasoning traces during streaming. Despite this limitation. the results in Tab. I0 consistently show that combining reactive and planning paradigms improves performance across all levels of time pressure.
22

---

Published as a conference paper at ICLR 2026
Additionally, observe that Gemini-2.5. built-in budget control struggles to precisely reg- ulate token usage, often exceeding the allocated budget (Figure 99) This highlights the ongoing challenge that LLMs face in controlling computational costs during real-tim reasoning.
Table 10: Performance of Gemini-2.5-Flash (Medium-difficlty Freeway) under various agent designs. We implement budget control by setting the thinking budget equal to the time pressure budgets. Since Gemini-2.55Flashe
internal reasoning traces Thinker directly. Instead, we reduce Reactive + Planning to allowing reactive thread to reference the final non- non-thinking tokens produced by planning thread after its reasoning is completed.
public, we cannot apply Agile-
Reactive (thinking off)
Reactive
Planning (thinking on)
Reactive+ Planning
Tokens/Step
(thinking on + budget control)
32k
0.12
0,93
0.93
0.92
16k
0.12
0.76
0.70
0.70
8k
0.12
0,09
0.25
0.31
4k
0.12
0.00
0.05
0.26
25000
Within Budget
Over Budget
6.4%
Reactive (Thinking On) Token Usage
33.5%
20000
0.0%
15000
90.0%
10000
5000
4k Budget
8k Budget
Budget
32k Budget
Figure 9: To test the effectiveness of thinking budget control, set the thinking budget of Gemini- 2.5-Flash at 4k, 8k, 16k and 32k to investigate the distribution of response length. The distribution shows that even Gemini-2.5- 2.5-Flash is aware of the budget, it cannot precisely regulate response token count, often generating excessive tokens.
C.4 CODE-AS-POLIEE
We analyze generated code understand the limitations of code-as-policy approaches. Below representative examples from each environment. All code directly generated by DeepSeek-R1 with some helper functions omitted for brevity.
FREEWAY
def next_action (json__stttsss -> str: $ Extract game state functions
Helper
t0, yo, cars extract_game_state (json_sttootttt max_time t0 50
Reasonable
23

---

Published as a conference paper at ICLR 2026
time herizon visited set queue deque () start (t0, yo, [1) (
current_curn, (start)
y, patinactions) visited. (ttO, yo
while queue: # BPS path finding $ BFS found path: choose
safest
immediate next action...
Freeway has simple, efficient solution: breadth-first search (BFS) explore all possible move- ment sequences across lanes, and the state space is very small (time x player position). Therefore, LLMs implement naive search strategies to get optimal performance.
SNAKE
def next_action (json_state) str: 8 Extract game state Helper functions
Main logic: choose best move for current state safe_moves moves
nenerate_saaf__ovee (json_state) if not sate_moves: return 't' no safe move
nove None best_ score -float (' iin') depta Limited depth
search
for move in safe_moves: next_state, json state, move) [ecure_rewart search (next state, depth 1) total_reward
simulate_move
immediate_reward renterd
future_reward if total_rewaal > best_score: best_score
total_reward
best_move move
11
12
return best_move
Snake does not have an algorithmic solution with bounded complexity, forcing LLMs resort to heuristic approaches. The generated code typically implements limited-depth search (5 turns in this example) evaluate immediate rewards, this constrained horizon proves insufficient for complex scenarios. Long-term consequences such as distant food opportunities or collision risks that emerge beyond the search depth remain invisible to the policy.
OVERCOOKED
def next_action, (json state: dict) -> str: Extract game state $ Helper functions
alice_held_obj alice. (' ('held_objectt
Decision logic if alice_held _bj and alice alice_held_ott held_obj.get ('name') 'onion':
Find available pots and add ingredients
elif alice held and reacy soup elif not he $ Priority 1: Get plate for ready
get name") 'plate': & Pick up
soups # Priority 2: Get ingredients for cooking
11
return 'S' Stay as default
def next_action (json_state: dict) -> str: if Aiice_held_bbj name
') = 'onion'
pot fot add ind_iingeeie (pot. elif al ce__hee__eoo_________________e_e get name!)
'plate':
24

---

Published as a conference paper at ICLR 2026
soup find ready_soug pickup_ready soup elif not alice obj: Priority
1: Get plate for ready soups # Priority 2: set ingredients for
cooking return
g' Stay default
Overcooked has complex game context, demanding that players infer partner intentions through action history and plan coordinated responses based on current state. However, the generated code considers only limited context (e.g., what Alice is currently holding). This narrow focus leads to suboptimal decisions. instance, when the agent holds an onion but all pots are occupied (perhaps by Bob), it simply stays idle rather than dropping the onion to pick up plate and serve ready soup.
def next_action (json_stttaa iixt_aatia dict)
str: Extract game state
Heiper functions
pots [tuple (pos) for pos in layout Pot ready_soups [ob] for obj in objects
if (' name') =- 'ouu'' and obj.get (" ('is_ready', False)]
cocking_soupe
[obj for ebj in objects if obj. ('name') == 'ouup' and obj.get (" False)]
is_cooking',
alice_held_obf = alice. ('held_object''
Priority Serve ready soup ready_soupe: if alice_held_odd and biic_hell_ooj.ee ('name')
-- -'supp': Deliver soup to serving counter elif alice_held_obs and
alice held abjd_bbb.gettteeeeeeeee name') 'plate': Pick up ready soup from pot else:
14
Get plate from dispenser
Friority 2: Handle cooking soups clif cooking_ soup
cooking_soupp soups [0] and not alice_held_obj: # Get plate for
17
if soup remaining_cookinggtion
# Priority 3: Start cooking fadd ingredients to empty pot) pot next (pot for pot in pots if not any tebj. get position') -- 1ist (pot) for obj in objects)), None) if idle_pot: if held_obj and held_obj.ger ('name')
20
'onion Add ingredient to pot ... else: Get ingredient from dispenser
22
23
return 'g' Stay default
Beyond ignoring broad game context, the generated code exhibits inconsistent goal prioritization that leads to oscillatory behavior. Consider scenario where Alice holds an onion and ready soup is available: the first code prioritizes adding the onion to empty pot, while the second prioritizes serving the ready soup. These conflicting objectives cause the agent to alternate between incompat- ible actions. This demonstrates fundamental limitation of code-as-ppa onde-as-policy approaches: while they can encode local heuristics effectively, they struggle to maintain coherent long-term strategies in environments requiring dynamic coordination and context-aware decision making.
25

---

Published as a conference paper at ICLR 2026
C.5
PERFORMANCE UNDER LIMITED THROUGHPUT
Game
Parallel Threads
Concurrent Threads
Reactive (V3)
Planning (R1)
Freeway 0.84 Snake
0.72
0.32
0.22
0.54
0.58
0.49
0.03
Overcooked
0.92
0.85
0.37
0..9
Table 11: Performance of AgileThinker with abundant (parallel threads) and limited (concurrent threads) throughput.
While our main experiments assume parallel execution with independent throughput for each sub- system, practical deployments may face resource constraints. In this section, we investigate whether AgileThinker remains effective when LLM and LRM share computational resources through con- current switching rather than parallelism. Specifically, we evaluate the performance of agent systems under equivalent token throughput, implementing the AgileThinker alternating inference between reactive and planning thread (concurrent generation) rather than simultaneous generation.
Figure [1 compares concurrent AgileThinker against parallel AgileThinker and single-model base- lines. Even with equivalent throughput, concurrent Agile AgileThinker significantly outperform both reac- tive agents and planning agents. While parallel execution offers modest performance improvements over concurrent execution, the gains are relatively small compared to the fundamental advantage of AgileThinker. This indicates that the primary benefit stems from cognitive specialization rather than computational resources, and Agile remain effective for coootree-onstrriieed deployments.
D WALLTIME EXPERIMENTS
70-
Thinke 32k
AgilTTinker
AgieTTinker 8k
Reactive
Episode Time (K seconds)
Plannin 1sk
Planning 12k
Pisaning
Planning
40-
Litear Fit 0,9999)
30-
Slope: 0.0473
R": 0.999
250
500
1000
1250
1500
Episode (K)
Figure 10: Almost linear correlation between generated token count and wall-clock time using DeepSeek official API, demonstrating the validity of our token-as-time abstraction. Here the num- bers after agent methods, e.g. 4k, 8k, refer to the corresponding environment time pressure budgets.
E DYNAMIC ADJUSTMENT ALGORITHM FOR INTERNAL BUDGET
Inspired by the AIMD algorithm we conduct additional experiments to show that thread resource be allocated adaptively through simple dynamic adjustment mechanism for Ny Specifically,
Phttps://ent wik wtt.:/iwiiiiii..ggooooa. /tddiiiiiiiiieee vive_ www.easeemmltaa..... htt.a//eeeivviieeeeeeeeeeemellla
iipl icative_
decrease
26

---

Published as a conference paper at ICLR 2026
initialize Nrx to small value (Ik tokens). If the reactive thread fails to generate complete response within the allocated tokens, we increase Nrx by 1k tokens. Conversely, if the reactive thread uses fewer than tokens for 3 consecutive responses, decrease Nr.. by 0.5k tokens. This approach eliminates the need to predefine fixed NTR, allowing the model to adapt based on query complexity.
In fact, the advantage of AgileThinker still significant with this adaptive algorithm, as we can see in Tab. 12 The experiments conducted DeepSeek-Vas V3.2 under Hard cognitive load and 8k tokens/step time pressure.
Algorithm Dynamic Budget Adjustment
Nr <- 1024, efficient streak <-
2: while game not terminates do
3: Execute AgileThinker with internal budget NTx
4: if reactive thread exceeds budget then
5: NTw + NIR + 1024
6:
efficient.streak +-
7: else
8:
efficient streak efficient streak efficient_streak 2 then
9:
NTx max [024, Nrx 512)
11:
efficient streak +
end if
13:
end if end while
14:
Reactive (Non-thinking)
Planning (Thinking)
AgileThinker (fixed, optimal NTw)
AgileThinker (dynamic Nrw)
Environment
Freeway
0.4659
0.1025
0.6352 (3k)
0.6279
Snake
0.2556
0.0550
0.3456 (Ik)
0.4166
Overcooked
0.5246
0.0134
0.6563 (2k)
0.6741
Table 12: Scores of AgileThinker under dynamic internal budget.
27


Reviews:
[{"review_id": "kLuLrECCcC", "reviewer": "Reviewer_TV6c", "summary": "This paper addresses the limitation of existing LLM-based agents in dynamic environments (where environments evolve during agent reasoning) by formulating \"real-time reasoning\" as a new problem and building Real-Time Reasoning Gym (with Freeway, Snake, Overcooked) to evaluate it. It studies two agent paradigms—reactive agents (bounded computation for speed) and planning agents (extended reasoning for complexity)—finds their flaws, and proposes AgileThinker, a dual-thread model that combines both by letting the reactive thread use partial planning traces. Experiments show AgileThinker outperforms single-paradigm agents as cognitive load and time pressure increase, and wall-clock tests confirm the linear correlation between tokens and real inference time, verifying its practicality.", "questions": "In terms of evaluation tools, what unique contributions does the Real-Time Reasoning Gym (with three games: Freeway, Snake, Overcooked) bring to the field of real-time agent evaluation?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. It innovatively defines the \"real-time reasoning\" problem for LLM agents (addressing the flaw that environments evolve parallel to agent reasoning) and builds Real-Time Reasoning Gym with three games (Freeway, Snake, Overcooked) to systematically control cognitive load and time pressure (using tokens as a hardware-agnostic time proxy).\n2. The proposed AgileThinker has an innovative dual-thread design: its reactive thread references partial planning traces for real-time decisions, solving the limitations of existing dual-system methods (independent operation or waiting for completion), and experiments with the same model family verify its superiority over single-paradigm agents.\n3. It emphasizes practicality: wall-clock experiments confirm a strong linear correlation between token count and real inference time (R²=0.9986).", "weaknesses": "1. AgileThinker lacks an adaptive mechanism for thread resource allocation: the optimal token budget for the reactive thread ( $N_{TR}$ ) varies across environments (e.g., ~5k tokens for Freeway vs. ~2k tokens for Snake/Overcooked) and requires manual empirical tuning, with no solution proposed to dynamically adjust it based on real-time environmental changes.\n2. Experimental scenarios are disconnected from real-world complexity: all experiments are conducted on three simulated games (Freeway, Snake, Overcooked), and there is no verification of AgileThinker’s performance in real-world complex scenarios (e.g., SWE-Bench, BrowseComp), weakening the practical reference value of the results.\n3. The contribution is unclear. Existing works [1] control whether models conduct deep thinking through mode switching; what is the essential difference between AgileThinker and these works?\n\n[1] Wu, S., Xie, J., Zhang, Y., Chen, A., Zhang, K., Su, Y., & Xiao, Y. (2025). ARM: Adaptive Reasoning Model. ArXiv, abs/2505.20258.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8KW7zPwqrU", "reviewer": "Reviewer_hZPr", "summary": "This paper introduces the real-time decision-making into LLM based agents. Instead of having an agent-environment loop, the proposed real-time reasoning gym imposes a time pressure on agent reasoning manifested in the form of token limits, and tests how well agents can complete tasks under the pressure of bounded reasoning. The proposed gym features three diverse synthetic agent environments with different challenges that arise from dynamic environments, and experiments show that previous LLM agents perform poorly on these challenges, especially when time-pressure is high, sheding light on a new direction of research.", "questions": "1. Table 2 and Section 6: what's the metric shown here, and what's the wallclock limit for these agents?\n2. How does the proposed framework generalize to cases where the environment updates at a cadence that is not predetermined, e.g., one update takes 1s and another takes 10s?\n3. How does AgileThinker generalize to other agent designs?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Real-time reasoning of AI agents under time pressure is an under-evaluated direction in the current wave of AI agents research, and this paper presents a good setting and three diverse synthetic environments to prototype research in this direction. Experiments show that with increased time pressure and task complexity, existing agent performance decrease significantly from the case where the environment waits indefinitely for agent reasoning.\n\n2. The paper presents AgileThinker, an agent design that combines a planner and a reactive agent in a manner that controls planner output with a token budget for the agent to stay under the overall allotted time pressure limit. This agent design, albeit simple, shows gains over purely planning or reactive agents especially when high time pressure is applied on the agent.", "weaknesses": "1. The assumption that token limit is a good proxy for actual wallclock time is valid only when all agents are implemented with the same LLM served with the same hardware. Even if hardware independence is desirable, it is unlikely that different models will share the same slope (TPOT) and intercept (whatever factor that goes into time-to-first-token), because of model size, model architecture, etc. The paper's formulation and experiments neglected this important consideration. \n\n2. As also seen in the result in Section 6, one crucial factor the token budget limitation neglects is the time-to-first-token cost of LLM calls in these language agents. Contrary to what the authors lead the reader to believe, the step time in the results presented in the paper for these simulated games is not 100s of milliseconds (as those familiar with these games would believe), but minutes (5.5 minutes to be precise). This is an important caveat, and undermines the real-time nature of the gym itself, since the environment is still unduely waiting for the agent to perform indefinite internal operations.\n\n3. The paper could do a better job contextualizing itself in the literature.\n\n    1. There is a limited set of agent designs that this paper experimented with, especially open-source implementations of agents from prior work to help the reader understand whether the baselines used in this paper are valid or competitive, and more importantly, whether the proposed AgileThinker framework can be a drop-in improvement for existing agent implementations.\n    2. Real-time agents have long been a subject of AI research, from best-first search (A* and variants) and robotics [1], to more recently real-time customer support AI agents [2]. The paper could use a better review of related work in this direction, and better distinguish the key contributions it's making. The concept might not be entirely new, but it's useful to call it out in the new paradigm of AI agent design and evaluation.\n\n[1] OpenVLA: An Open-Source Vision-Language-Action Model. (https://arxiv.org/pdf/2406.09246)\n\n[2] Asynchronous Tool Usage for Real-Time Agents (https://arxiv.org/pdf/2410.21620)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nHF4hpJm9O", "reviewer": "Reviewer_sdJh", "summary": "The authors propose Real-Time Reasoning Gym, an environment consisting of three games (Freeway, Snake, and Overcooked) that are designed to test LLM agents in dynamic environments. Dynamic environments are those where the environment progresses at a fixed rate regardless of whether the agent has been able to come up with an action or not. To evaluate how current methods perform in this new setting, the authors study both reactive agents (non-thinking models, budget forcing) as well as planning agents (thinking models, code as policies). They find that each type of agent has its limitations, motivating the novel AgileThinker agent which runs two parallel threads (one for reactive, one for thinking) and mostly matches or outperforms all baselines in all settings.", "questions": "1. Figure 3: what does “reference” mean? And why is there two frames marked as “Reactive Agent Output” under the Static setting? I thought the agent only outputs actions, but here somehow the observations seem to be marked as output?\n2. Page 4: “For each game, we normalize the scores by the highest score the agent could get in that game, so we always have a score between 0 and 1.” → could the authors clarify exactly how this normalization works maybe by giving an example?\n3. Page 5: “(2) code plans where a thinking model is used to generate a code snippet that automatically produces actions based on observation input (Liang et al., 2022; Zhang et al., 2025). Although (1) is often easier to generate, (2) is more adaptive to potential changes.” → could the authors explain this a bit more? As in, why is generating code plans more adaptive to potential changes? Also, it’s not immediately obvious to me that generating code should fall under planning?\n4. Figure 6: for the “Planning Reasoning” yellow box in the middle, why is there only two apples instead of 3? In all other cases there seems to be three.\n5. Figure 6: is this a case study of real trajectories?\n6. Figure 5: it seems that in the bottom right plot, AgileThinker does better than the planning methods even though the time pressure is very lenient. I would have expected these methods to be at best on par here, but somehow AgileThinker outperforms the planning methods - could the authors comment on why this might be the case?\n7. Page 8: “we also plot the cumulative distribution function (CDF) of R’s token usage across all game trajectories without constraints” → does this mean for the purpose of creating the CDF, R gets to see all of P? Or how is the partial reasoning reference point determined if there is no constraints?\n8. Table 2: what’s the cognitive load setting for these experiments?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. The work provides a very novel and interesting problem setting for language agents, lifting the unrealistic assumption that the environment waits for agents to execute their actions.\n2. The motivation is clear and overall the paper is well-written\n3. The quantitative results are very strong, and the authors additionally provide a qualitative case study further illustrating the benefits of AgileThinker.", "weaknesses": "I could not discern any significant weaknesses, but I do have some questions in a few places (see questions section).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "m2TXkEmgr6", "reviewer": "Reviewer_EhbH", "summary": "The paper introduces AgileThinker, a dual-thread architecture for LLM agents that must act while reasoning in dynamic environments. It defines the real-time reasoning problem, where environments are dynamic and the agent’s computation time directly affects outcomes. Using the new Real-Time Reasoning Gym benchmark, the authors show that combining a fast reactive thread (for immediate responses) with a slower planning thread (for long-term reasoning) enables a better balance between speed and deliberation. Experiments demonstrate that AgileThinker outperforms purely reactive or planning agents under varying cognitive load and time pressure, with results validated both in simulated and real wall-clock settings.", "questions": "1) What are some alternatives to TR, i.e., alternatives to fine-tuning this hyperparameter for each domain/task?\n2) What are the foreseeable computational and latency overheads of running two LLM threads in real-world systems, especially under networked or resource-constrained deployment?\n3) How does AgileThinker compare with Yaoru et al.? Unless there are strong reasons not to, this is a good baseline to test AgileThinker.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1) The paper introduces real-time reasoning as a new challenge for LLM agents and proposes AgileThinker, a dual-thread framework that combines fast, reactive behaviour with slow planning. This is a clear conceptual advance beyond existing static or turn-based agent paradigms.\n2) The authors validate their approach through a carefully designed benchmark (Real-Time Reasoning Gym), multi-game testing, controlled ablations, and statistical significance tests (appendix). The experimental evidence is strong and aligns with the theoretical motivation. \n3) The paper is well-written, well-illustrated, and accessible, effectively communicating complex timing and coordination concepts. I found the work really interesting.", "weaknesses": "1) All experiments use simulated game-like settings (Freeway, Snake, Overcooked); they may not fully represent complex real-world tasks (e.g., robotics, human-AI interaction). The results demonstrate concept validity but not necessarily scalability to multimodal or continuous-control domains.\n2) The paper needs to explore comparable baselines like: \n   - Li, Yaoru, Shunyu Liu, Tongya Zheng, and Mingli Song. 2025. “Parallelized Planning-Acting for Efficient LLM-Based Multi-Agent Systems.” arXiv [Cs.AI]. arXiv. http://arxiv.org/abs/2503.03505.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)