Test-Time Adaptation for LLM Agents via Environment Interaction
Authors: Arthur Chen
Zuxin Liu
Jianguo Zhang
Akshara Prabhakar
Zhiwei Liu
Shelby Heinecke
Silvio Savarese
Victor Zhong
Caiming Xiong
Venue: ICLR 2026 Poster (2026)
Keywords: Large Language Model
Agent
Test-Time Adaptation
Primary area: applications to robotics, autonomy, planning
TL;DR: We propose a framework named grounded test-time adaptation to adapt LLM-based agents to novel and complex environments.

Abstract: Large language model (LLM)-based agents struggle to generalize to novel and complex environments, such as unseen websites or new sets of functions, due to a fundamental mismatch between their pre-training and test-time conditions.
This challenge stems from two distinct failure modes: a syntactic misunderstanding of environment-specific components like observation formats, and a semantic misunderstanding of state-transition dynamics, which are only revealed at test time.
To address these issues, we propose two distinct strategies for adapting LLM agents by leveraging environment-specific information from interaction that is available during deployment.
First, an online syntactic alignment (SA) method parameterizes environmental nuances by learning a lightweight adaptation vector that biases the model's output distribution, enabling rapid alignment with an environment response format.
Second, a deployment-time dynamics grounding (DG) method employs a persona-driven exploration phase to systematically probe and learn the environment's causal dynamics before task execution, equipping the agent with an in-context world model.
We evaluate these strategies across diverse agentic benchmarks, including function calling and web navigation.
Our empirical results show the effectiveness of both strategies across all benchmarks with minimal computational cost.
We find that dynamics grounding is particularly effective in complex environments where unpredictable dynamics pose a major obstacle, demonstrating a robust path toward more generalizable and capable LLM-based agents.
For example, on the WebArena multi-site split, this method increases the agent's success rate from 2\% to 23\%.
We release our code.

Published as a conference paper at ICLR 2026
TEST-TIME ADAPTATION FOR LLM AGENTS VIA EN- VIRONMENT INTERACTION
Arthur Chen'
Zuxin Liu?
Jianguo Zhang
Akshara Prabhakar²
Zhiwei Liu?
Shelby Heinecke Silvio Savarese
Victor Zhong't
Caiming Xiong2t
University of Waterloo
Salesforce Al Research
ABSTRACT
Large language model (LLM)-based agents struggle to generalize to novel and complex environments, such unseen websites or new sets of functions, due a fundamental mismatch between their pre-training and test-time conditions. This challenge stems from two distinct failure modes: syntactic misunderstanding of wwvironment-speciffe components like observation formats, and a semantic misun- derstanding of state-transition dynamics, which are only revealed test time. To address these issues, we propose two distinct strategies for adapting L.LM agents by leveraging environment-speee during deployment. First, an online syntactic alignment (SA) method parameter- izes environmental nuances by learning a lightweight adaptation vector that biases the model's output distribution, enabling rapid alignment with an environment response format. Second, deployment- -immeettttteeeeteee dynamics grounding (DG) method employs persona-driven exploration phase to systematically probe and learn the environment's causal dynamies before task execution, equipping the agent with in-context world model. We evaluate these strategies across diverse agentic benchmarks, including function calling and web navigation. Our empirical re- sults show the effectiveness of both strategies across all benchmarks with minimal computational cost. We find that dynamics grounding is particularly effective in complex environments where unpredictable dynamics pose a major obstacle, demonstrating a robust path toward more generalizable and capable LLM-based agents. For example, on the WebArena multi-site split, this method increases the agent's ' success rate from 2% to 23%. We release our code
information from interaction that is available
INTRODUCTION
Large language model (bLMe Al agents have demonstrated remarkable capabilities across diverse tasks (Li et al. 2023a Zhou et al. 2024a Patil et al., Xie et al. 22024), yet they often fail novel agentic environments such as web navigation (Zhou et al., 2024a] He et al. 2024) and function calling (Patil et al., Yao et al.) 2024). These failures reflect two core challenges- the systematic mismatch between the agent's 's pre-trained knowledge and the specific syntax and dynamics of the deployed environment (Yang et al.) 2025; Thil et al. 2024). Although both mismatches stem from a lack of prior knowledge, we distinguish them operationally because they require different adaptation mechanisms (syntactic vs. semantic). To illustrate these two failure patterns, we consider an LLM agent tasked with booking a flight on a previously unseen travel website:
generalize when deployed
1. Syntactic Mismatch: the LLM agent's prior knowledge does not align with environment-specien information, such as observation structure (Yang et al. (2024] Gur et 2023) and the environ- ment's unique syntax (Lei al., 2024 Chen et al.] 2024). This causes parsing and context- understanding issues: a pre-trained LLM agent might try cl ilcck ( 'Search' " or target
Work done during internship. Correspondence hacnan.
Equal advising.
https: thub. http://..m//mmm//111111

---

Published as a conference paper at ICLR 2026
destination. while the site exposes Go and dest sest_-feedd producing invalid actions
and immediate failures. (Yang et al.) 2024: Gur et al. 2023 Lei et al. 2024 Chen et al. 2024)
Semantic Mismatch: the agent lacks an accurate, environment-specifee causal model of state transitions, so it cannot predict the consequences of actions. For example, the agent may expect click Go" to show flight results, but the site instead opens a date-confirmation pop-up: without knowing this transition, the agent cannot form the correct multi-step plan and fails. (For-
rester 1971; Chae et al. 2025 Zhang et al. 2024)
Current approaches to this generalization gap ill-suited for rapid adaptation in novel envi- ronments. In addition, many methods require human-annotated or LLLM-anntttted demonstra- tions (Wang et al., 2024b, Murty et al. 2025; Xu et al. 2024), which can be resource-intensive or rely on LLM's prior knowledge about the environment. On the other hand, to address semantic mismatch, explicit world modeling like (Chae et al. 2025 involves heavyweight pipeline of exten- sive data collection and fine-tuning tuning separate, specialized model that is computationally expensive and struggles to generalize without being retrained. Both paradigms present significant overhead, highlighting the need for more efficient strategies that can ground an agent using only the informa- tion available at test time.
To address the two failure modes and close the generalization gap. we propose two annotation- free strategies that operate at deployment time. (1) Online syntactic alignment (SA)-a lightweight module that learns a small per-interaction bias (an adaptation vector applied to late features) to quickly align an agent's 's output distribution with anviromeent-specife syntax. Observing Go and dest field on the page. for example, lets the adapter steer action generation toward the site's actual element names. (2) Dynamics grounding (DG)-a short, persona-guided exploration phase that iteratively collects in-context rules about state transitions (e.g., clicking Go opens date pop-up) and supplies those rules as context for downstream planning. Both methods require only test-time observations and no trajectory annotations or expensive fine-tuning.
We evaluate our methods on function-calling and web-navigation benchmarks. Both approaches yield consistent improvements: syntactic alignment gives efficient, steady gains across sites, while dynamics grounding produces particularly large improvements on environments with unfamiliar transitions. For example, on the WebArena multi-site split (Zhou et al.) 2024a), dynamics ground- ing raises GPT-4. success from 2% to 23%. Taken together, syntactic and semantic test-time grounding substantially narrow the deployment generalization gap and offer a practical path toward more robust LLM agents.
2 RELATED WORK
Test- Adaptation Originating in computer vision, test-time adaptation (TTA) addresses dis- tributional shifts between training and testing data by adapting models inference time (Wang et al.] 2021 Niu et al. [022;] Sun et al. 2020). Our syntactic alignment builds directly methods that up- date model parameters or steering vectors at test time to minimize an unsupervised objective, such as entropy minimization or self-supervised learning (Wang et al. 2021 Hu et al. 2025 Li et al., 2023b). Our syntactic alignment aligns with steering vector research (Tan et al. 2025] Sinii et al, 2025 Panickssery et al. 2024 Turner al. 2024) in its use of latent space biases to shift output distributions. However, standard steering approaches generally apply modulation vectors to shift high-level traits (Tan al., 2025). , such honesty (Zou et al. 2025). We distinguish our work by tailoring this mechanism to the dynamic nature of agentic tasks: our vector is treated as tempo- rary parameter, updated online using the environment's response
self-supervisory signal, and reset per episode to ensure precise, context-dependent alignment without interference across tasks. While TTA for LLMs is emerging for tasks like math reasoning (Zuo al., 2025) and few-shot learning (Akyürek et al., 2025), its application to the unique, interactive, and stateful challenges of I.IM agents remains underexplored. Our work the first to systematically apply this paradigm to adapt agents to novel environment formats without supervised trajectories.
Environment Modeling for Agents Recent work has focused on incorporating environment dy- namics into LLMebbsed planning by training a parameterized world model (Fang et al.. 2025| Chae ct al. 2025 Ding et al.) 2026) Qiao et al. (024). or using an LLM to predict the next state using continuously updated environment nules (Zhou al. 2024b). Such approaches typically require
2

---

Published as a conference paper at ICLR 2026
collecting large set of observations for learning world model, and can be computationally expen- sive (Chae et al. 2025). In contrast, our dynamics grounding introduces a lightweight, deployment- time pipeline generate temporary, in-context world model from few exploratory interactions. This aligns with a broader trend of using in-context learning for adaptation, including methods like agent workflow memory (Wang al. 2024b) and retrieval-augmenned -agmentd generation (RAG) for plan- ning (Luo et al. 2023; Kagaya et al., 2024; Wang et al., 2024a). However, our contribution lies in the systematic and automated process of generating this knowledge via persona-driven exploration, removing the need for pre-exisee pre-existing annotated demonstrations separate, trained dynamics model.
LLM-Based Agents Large language models (LLMs) have become the planning backbone of a wowpenetttooooooliinnneeoo following capabilities and generaliza- tion across diverse tasks (Ouyang et al. 2022: Mishra et al.] 2022 Zhou et al. 2024a). Their ability
wide range of agentic systems due to their strong instruction
to interpret complex instructions and generate coherent action plans has enabled their integration into applications such as web navigation (Zhou et al.
2024a He et al. 2024, Lu et al.] 2024 Deng
al. 2023) and function calling (Patil al.) Yao al.
2024). Despite recent advances, LLM agents
struggle to comprehend the specific syntactical input structure of different environments, such as in
web navigation (Yang et al. 2024 Gur et al., 2023) and text-to-SQL (Lei al., (024). This is
due to the fundamental mismatch between the LLM pre-training corpus and the specific agentic
environment (Chen et al.
2024 Ben-David et 2010
GROUNDED TEST- ADAPTATION STRATEGIES
In this section, we detail our two adaptation strategies which mitigate syntactic and semantic gaps. We first formalize the problem setup, which is designed to reflect realistic deployment scenarios and defined by three key constraints:
1. No annotated trajectories offline data. The agent cannot access expert demonstrations or pre-collected data from the target environment. It must learn from scratch during interaction. For our running travel example, the agent has never seen wwweelexample.com and has no pre-existing dataset of successful bookings on that site.
Online, streaming adaptation. The agent receives tasks one at a time and must adapt on-the-fly without sceing a full batch of test instances in advance. For example, must adapt to the web- site's 's quirks while handling a single request (e.g. "Book a flight from New York to Saskatoon") before seeing any future tasks.
3. Permitted test-time interaction. The agent is allowed to interact with the environment gather information before receiving specific task, but without supervision. For instance, it can perform "blind" exploration of travel-example.co com discover that button triggers calendar pop-up, but it does not know what kind of tasks the user will ultimately ask it to execute.
This setting mirrors constraints in practical benchmarks (e.g., BIRD (i et al,, 2023a). We- bArena (Zhou et 2024a)) and necessitates methods that can leverage only unsupervised, test-time interaction. To formulate this challenge, at test-time the LLM input Z is constructed as
I = [p; {-)
(1)
where p denotes the task instruction (e.g., rules, task description), o denotes the current environment observation (e.g., accessibility tree observation, response of function call), and {a) denotes the sequence of past actions taken by the LLM-based agent up the current step. For environments with short or simple observations (e.g., tool-use scenarios), we instead construct the model input using all previous interactions between the agent and the environment, which may include both en- vironment responses and agent actions. With this formulation of the challenge in mind, we describe two adaptation methods in the remainder of this section.
3.1 SYNTACTIC ALIGNMENT
To address syntactic mismatches that arise in novel environments, propose syntactic alignment. The objective is to parametrically adapt the model's output distribution at test time by treating the current context (i.e. task instructions, observations, and action history) self-supervisory signal.
3

---

Published as a conference paper at ICLR 2026
DIVIRONMENT INPUTS
ADAPTATION LEARNING
Model puts
Input Construction
You
tacked with son gatting browclin
ligerr
STAGE
STABE
109ooe soucea tike-
Agentic Environments
STEP lapats
Lask Instuchors
P
clemer)
action clibks
Undate Para nefere
sharvations
[64] lrk HP CSNSAAAAAE
(kanowod)..
A1
Action
STEP 2: Leans
Now Action lsken with byc
Pad aclere
Adogiaaiin
I
Foodback Loop
Figure 1: Overview of syntactic alignment (SA). This figure includes an example of web nav- igation shopping task to illustrate how the agent adapts new environment. (1) At the start of each episode, we initialize an adaptation vector as zero vector and construct inputs to the LLM agent. (2) During task execution, the agent receives environment instructions and observations. (3) At each step, we update the adaptation vector using cross-entropy loss on the current input, and apply the adaptation vector as bias to the LLM's final hidden layer. This enables rapid alignment to environment--ppecife observation and action formats. (4) The LLM agent takes new action with the updated vector, which shifts the model's output distribution to better match the test-time environment.
This allows the model to align with local syntactic patterns, such as unique UI element labels or response formats, without requiring a deep understanding of the environment's 's causal dynamics.
Our approach introduces single lightweight adaptation vector 5 IR where d is the hidden dimension of the language model. This vector acts as an additive bias to the final hidden representa- tions H Raad before the final projection layer. At each step of an episode, the adapted logits computed as:
logits' = (H + W)
2)
where WLM E RIV the model's output projection matrix and [V is the vocabulary size.
This vector is updated at each turn by performing one gradient descent step to minimize the language modeling loss of the current input context according to the following:
&new < 8old - 5LE
I2:n)
(3)
where fo,8 the LLM parameterized by its fixed weights 0 and the adaptable vector 5, and v is the learning rate. Ii--1 denotes the input subsequence consisting of tokens 1 through and Iz. denotes the corresponding next-token targets obtained by shifting the sequence by one position. The loss in Eq. 3s therefore computed by predicting each token in Iz.n from its preceding context I This update encourages the model to internalize the syntactic features present the immediate context by adapting its output distribution to the environment context.
1,
To illustrate this in practice, consider our agent on the unseen website travel exxmple com. The agent's imput context I contains the accessibility tree, which includes strings such [145] <button>Co and dest.ffell.
1. Initialization: At the episode's start, the adaptation vector is initialized
vector, 8 <- 0. Without adaptation, the model's logits would likely favor the action click Search"' based on pre-trained priors.
Prediction:
3. Adaptation: We compute the language modeling loss on the context I (Eq. 3 Because the strings Go and dest_field are present in I, the gradient update to 5 will shift its values to favor their tokens.
4. Correction: In the subsequent generation step, the adapted logits (Eq. now assign a higher probability the syntactically correct action, click "o"
This process is computationally efficient, and steps 2-4 are repeated at every model prediction within an episode. To prevent catastrophic forgetting and ensure that adaptation is specific to the currett
4

---

Published as a conference paper at ICLR 2026
environment, 8 is reset to zero at the beginning of each new episode. This lightweight update aligns the model's pre-trained distribution with the specific distribution of the environment.
3.2
DYNAMICS GROUNDING
STAGE 1: ENVIRONMENT & GOALS
STAGE AGENT EXPLORATION
STAGE 4: FINAL MODEL
Agentic Environments
STEP 2: Goal-Guided Exploration
Final Env Dynamics
STEP Synthesize Goals
Clicking "Under
$10" will navigate product page with small gifts such including books and pens
Goal Synthesis
Observation Trajectory
STAGE 3: DYNAMICS LEARNING
STAGE FINAL MODEL
Middle school child browsing navigate small such including
Persona:
STEP 3:
VALID DYNAMICS
STEP 4:
Summarize Changes
ACTION: Click "Under $10*
Filter Dynamics
Final Dynamics:
books pans.
RESULT:
product page with small gifts...
Clicking "Under $10" will navigate to product page with small gifts such including books and pens.
Code:
get, weaceeeelllooainnnn (location:
str), invoka the location: str) with code as the location argument
FILTERED DYNAMICS
ACTION: Click product page
RESULT: Display product info
Figure 2: Overview of dynamics grounding (DG). This figure includes an example of web navi-
gation shopping task to illustrate how the pipeline generates environment dynamics in language. (1)
We synthesize diverse exploration tasks based on personas using environment descriptions. (2) An
exploration agent interacts with the environment to collect interaction logs of state transitions. (3)
An LLM extracts and summarizes environment dynamics from these logs. (4) A reasoning model filters less informative rules, which are then used to augment the agent's context during evaluation, enabling more transition-awes aware decision making.
LLM-based agents often fail in novel environments due to semantic mismatch: they lack a "world model" to predict the outcomes of their actions (Ha & Schmidhuber 2018]] Chae et al.) 2025) To address this, we propose dynamics grounding, a strategy that proactively discovers the causal dynamics of environment before the agent encounters any user tasks during deployment. The objective is to construct an in-context world model using natural language by performing a one- time, automated exploration. The cost of this discovery phase is amortized over all subsequent tasks performed in that environment.
Our method follows four-step pipeline generate a concise set of environment dynamics, denoted as Ectean. for a given new environment.
Persona/Exploratoor Goal Synthesis. An LLM prompted with high-level description of the environment (e.g., website purpose, API documentation) to generate a set of N diverse, ex- ploratory "personas" or "goals". Personas frame open-ended tasks designed to probe for non- obvious interactions. For our t drravel-examp
com scenario, synthesized persona might be: "As a first-time user, I want to see what happens if I try to search for a flghtt without select- ing date.". We prefer personas over less-structured exploration policy (e.g. maximizing state novelty or coverage) to guide the agent to explore complex, multi-step interactions that a naive exploration strategy might miss, thus yielding more semantically rich dynamics.
2. Exploration and on-the-fly Dynamics Extraction For each persona or goal, an LLM agent interacts with the environment. The agent is instructed to take novel actions to maximize its discovery of new state transitions. After each action is executed and the system transitions into the next state, we take (observation, action, new observation) to summarize into a concise human- readable rule Immediately after, the list of generated rules {e}? is appended to the agent to encourage taking actions that have not been taken before. In our example, the agent would execute c ick ( 'Go"") ) and log the transition from the main page state where a calendar
5

---

Published as a conference paper at ICLR 2026
pop-up is active. Then the transition on travel-exampl dynamic: Executing the action click ( "Go") on the main page causes a calendar modal to appear for date selection.
would be extracted as the
3. Filtering and Consolidation. The set of all extracted dynamics, E = {e) is passed to a reasoning model. This model filters out trivial (e.g., "typing in text field shows text") or repetitive rules, producing final, clean set of dynamics, Edlcan.
During task execution at test time, we augment the input context I with discovered dynamics:
I' I; Ectean]
(4)
By leveraging the L.M's in-context learning capabilities, this explicit knowledge of the environ- ment's state-transition patterns guides the agent to make more informed and transition-aware deci- sions. In our example, with the calendar pop-up dynamic its context, the agent correctly antici- pates the outcome of its click and plans its subsequent actions accordingly, successfully navigating the semantic mismatch. Example prompts for exploration and dynamics extraction are shown in Appendix sections section A.2 and section
4 EXPERIMENTS
4.1 SETUP
We evaluate our approach on three realistic agentic benchmarks: BFCLv3 (Patil al.). WebArena (Zhou et (024a)) and Tau-Bench (Yao et al. (2024). which together cover both web navigation and conversational function calling tasks. As for baselines, we evalu- ate open-source Qwen2.5-14B-ae 55-4----ssstu (Qwen eet al. (025)) model and closed-source EPT-4.1. On WebArena, we additionally compare to the (WMA) agent from (Chae jet al., (025) as a baseline. This approach first collects exploration demonstrations, then trains a I.l 1 ruct (Grattafiori et al.] 2024) model predict the next state given an action (i.e., a learned world model). During evaluation, this world model is used provide next-state predictions to the agent on-the-fly, helping the agent reason about the consequences of its actions. In this comparison, we use the same GPT-4o mini as the original WMA experiments. Aside from the syntactic alignment and dynamics grounding experiments, we provide ablation studies in section 4.3
Table I: Number of tasks per website in the WebArena benchmark. The benchmark consists of six websites, including a multi- site category for tasks that require interact- ing across multiple websites (from the six sites), for a total of 812 tasks.
Website Number of Tasks
World-moddll-uggmene
Shopping
187
Shopping Admin
182
GitLab
180
Map
109
Reddit
106
Multi-site
48
Total
812
For all dynamics grounding experiments, we leverage GPT-4. as an exploration policy to extract environmental dynamics. For each environment, we conduct 10 exploration episodes to gather diverse set of dynamics. For environment dynamics filtering, we apply o to remove repetitive and trivial environment dynamics. We also include ablation studies on the number of exploration episodes and the effect of filtering in section 4.3
Before delving into the results, first introduce the benchmarks and describe the specific experi- mental setups used for experiments.
WebArena The WebArena benchmark consists of five self-hosted websites and total of 812 tasks. Many tasks require interacting across multiple websites, which poses additional challenges for LLM-based agents. For evaluation, WebArena employs a maatnnigatchinn-.. assess correctness, providing more stable and objective results compared to benchmarks using LLMs to score, such as (He et al.. 2024, Gou et al., 2025). For dynamics grounding, we first synthesize 10 personas via prompt instruction for each website using GPT-4o (OpenAl et al., 2024) and with seed personas from NNetNav (Murty al. 2025). Then for each persona, we roll-out GPT-4. agent to explore the environment guided by the persona description, with the exploration prompt
mechanism to
htttss://platffee openai http://ccc/////////////////

---

Published as a conference paper at ICLR 2026
Main results on WebArena, Web WebVoyager, BFCLv3, and Rau-Bench benchmarks. We report task success rates (%) for each model and adaptation method. For Tau-Bench, we average runs across 5 seeds and use custom more stable codebase. Both test-time adaptation strategies improve performance. "N/A" indicates not applicable- -WM requires training a web-specific world model on state transitions, which is not applicable to Tau-Bench (conversational only) or BFCLv3 (no explicit web-like state transition data); Dynamics grounding does not operate in conversational Tau- Bench as there is no explicit and fixed state transition rules in the environment.
Table 2:
Model
Task
WebArena
BFCLv3
Tau-Bench
Airline
Retail
GPT-4.1
30.0
55.5 64.0 (+8.5)
GPT-4.1 (+ (GD
35.0 (+5.0)
N/A
N/A
GPT-40 mini
12.0
GPT-4o mini (+ WMA)
13.5 (+1.5)
N/A
N/A
N/A
GPT-4o mini (+ DG)
18.0 (+6.0)
N/A
N/A
Swen2.5-14B-Insstn
17.0
18.5
21.6
43.3
Swwe2.5-114B-Inttne (+ SA)
18.0 (+1.0)
20.0 (+1.5)
25.2 (+3.6)
44.9 (+1.6)
Owen2.5-14B-Instnnn (+ DG)
20.0 (+3.0)
22.0 (+3.5)
N/A
N/A
Qwen2.5-14B-Inntruuc (hybrid)
21.0 (+4.0)
21.0 (+2.5)
N/A
N/A
instruction Irnittioonerriinee derived from NNetNav, and with maximum exploration budget of 30 steps. To extract environment dynamics from interaction logs, we summarize each state transition pair (observation, action taken, new observation) with prompt instruction 3 and concatenate all summarized state transition pairs to obtain the list of environmental dynamics E = { :=1 for each website. Finally, utilize the o3 model remove repetitive and trivial environment dynamic entries from the list via prompt instruction 4 resulting in Eekcan. We provide all prompts we use in section A With syntactic alignment, we reset the adaptation vector after each episode and we use a learning rate of 0.1 and training step of 1 for all experiments. During evaluation, we use the text-based accessibility tree as the observation type and run experiments using BrowserGym of 30.
and and AgentLat Lant with a maximum step
BFCLv3 BFCLv3 (Patil al.) is benchmark with various realistic APIs to evaluate the functions calling abilities of L.MM. It contains 8 different API domains. We provide the number of APIs for each environment in table 8 We choose the multi-turn-base split to assess the agent's multi-turn function calling capabilities. This split is multi-turn and multi-step, meaning that instead of just handling a single tool call in one-off request, models have to carry out sequences of function calls across multiple dialogue turns. BFCLv3 uses baste evaluation (checking the system's 's resulting state) and response-basee evaluation (checking whether the call sequences are valid and minimally sufficient) to assess correctness, therefore being stable and deterministic. For dynamics grounding, we synthesize 10 exploration tasks with GPT-40 conditioned on the function documentation for each domain via prompt instruction 7 Concretely, we instruct the LLM to think about what functions wants to explore in order to find out unpredictable outcomes or interesting behaviors, given the function documentation. We use GPT-4 1 agent to execute the exploration task and we collect interaction logs for environment dynamics extraction. During exploration, we ask the exploration agent to decide if the steps taken far are sufficient to answer the exploration task with prompt instruction g The environment dynamics are extracted using prompt instruction g We follow the same procedure in WebArena to remove repetitive and trivial environment dynamics with prompt instruction I0 For syntactic alignment, we reset adaptation vector at each turn (when a new user query arrives).
Tau-Bench Tau-Bench (Yao et al., 2024) is a realistic function calling benchmark that simulates real-world service agents interacting with an LLM-simulated user. It comprises two domains: air- line (50 tasks) and retail (115 tasks). The airline domain is generally more challenging than retail, as it features much larger database, increasing the complexity of the tasks. Tau-bench 's use of an LLM-simulated user agent introduces significant variance in evaluation results, as the simulated hu-
7

---

Published as a conference paper at ICLR 2026
Table 3: Success rates (%) on the WebArena benchmark across different websites and models. Both adaptation strategies improve performance over basclines. and dynamics grounding (DG) surpasses WMA on GPT-4o mini. DG improves success rate substantially complex multi-site split.
Website
Model
Gitlab
Shop Admin
Shop
Reddit
Map
Multi
Avg
GPT-4,1
48.0%
31.0%
37.0%
29.0%
28.0%
2.0%
30.0%
GPT-4.1 (+DG)
45.0%
27.0%
33.0%
42.0%
31.0%
23.0%
35.0%
GPT-4o mini
9.0%
12.0%
18.0%
8.0%
12.0%
12.0%
12.0%
GPT-4o mini (+WMA)
13.5%
GPT-4o mini (+DG)
18.0%
14.0%
26.0%
14.0%
19.0%
12.0%
18.0%
Qwen2.5-14D5 1---141-Innttrccc
20.0%
14.0%
22.0%
13.0%
13.0%
6.0%
17.0%
Qwen2.5-14B 1-nn2-5--4B-Instruee (+SA)
16.0%
14.0%
24.0%
25.0%
16.0%
6.0%
18.0%
Qwen2.5- 1----14-Inntttctc (+DG)
23.0%
13.0%
24.0%
23.0%
20.0%
10.0%
20.0%
Qwen2.5- 14B-11-trenne (Hybrid)
26.0%
13.0%
24.0%
31.0%
19.0%
4.0%
21.0%
man may drift from the original instruction be influenced by the agent's responses over multiple conversational turns. To address this, we adopt the improved evaluation codebase from (Prabhakar et al. 2025), which substantially reduces reward variance and improves evaluation fidelity. Specif- ically, we (1) fix incorrect labels, (2) set the temperature of the LLM-simulated user agent to for deterministic responses, and (3) employ Best-of-N (N=5) sampling strategy combined with a self-critique mechanism, allowing the simulated user to better adhere to the task instruction and avoid being misled by the agent's outputs. This protocol has been shown improve consistency in agent performance evaluation across multiple trials according to (Prabhakar et al.) 2025). Tau- Bench consists solely of API-based tasks without observable state transitions, meaning there are no environment dynamics to extract. As result, dynamics grounding, which relies on extracting and leveraging such dynamics, is not applicable to this benchmark. Therefore, we only evaluate the syntactic alignment method on Yau-Bench. To further reduce variance, we report results averaged over random seeds in table
4.2 RESULTS AND ANALYSIS
Both Adaptation Strategies Consistently Improve Performance As shown in table both syn- tactic alignment (SA) and dynamics grounding (DG) improve performance across models and tasks. dynamics grounding yields substantial improvements for GPT-4. 1, which is better able to leverage the provided dynamies due its superior instruction-follooingg abilities. is also evident in table that our GPT-4o mini surpasses WMA that uses a learned world model by large margin. For Owen2. 1--1B-IInstrutt. both methods provide gains, highlighting the general applicability of test-time adaptation. The improvements of dynamics grounding are especially significant for models with stronger foolloiion-foloowing abilities (e.g., GPT-4.1,, likely because they can better leverage the provided dynamics context.
Both Adaptation Strategies are computationally efficient The two proposed strategies are computationally efficient, though they have different profiles. syntactic alignment is highly efficient for real-time use. As shown in table a single update step adds only a 3% latency overhead. In contrast, dynamics ground- ing is a one-time, deployment-time investment designed to be far more lightweight than traditional world-modeling approaches like the WMA baseline. For example, while WMA requires syn- thesizing 870 tasks to collect trajectories for training a separate Llama-3.. 3. 1-BB model, our method uses only 50 exploratory roll-
Table 4: Relative latency gain
(%) of PA on BFCLv3 multi- turn. LR fixed 0.1.
Step
Latency Gain
0%
1
3.0%
2
5.9%
3
8.2%
12.6%
5
15.6%
outs and requires no additional model training. This streamlined
process avoids WMA's costly training phase and its significant inference overhead, where action simulation takes approximately 140 seconds per step. The total upfront cost for our dynamics grounding procedure on single WebArena website is approximately 7M tokens, cost that is amortized across all subsequent tasks. This investment pays significant dividends where environ- mental dynamics unpredictable. For instance, the challenging multi-site split, the success rate of our agent increases from 2% to 23%. This demonstrates the value of efficiently learning an environment's dynamics at test time to compensate for the limitations of an agent's pre-training.
8

---

Published as a conference paper at ICLR 2026
Table 5: Ablation exploration policy and dynamics extractor backbones WebArena. Here we use different backbones of exploration policy and dynamics extractor. We found for dynamics grounding- using the same LLM agent (itself) improves the same using stronger one.
Task Model
Exploration Policy
Dynamics Extractor
SR
GPT-4o mini (baseline)
12.0
GPT-4o mini
GPT-4.1
GPT-4.1
18.0 (+6.0)
GPT-4o mini (self)
GPT-4o mini
GPT-4o mini
19.0 (47.0)
Swen2.5-14B-Instruue (baseline)
17.0
Owen2.5-144BInenn Swen2.5-14B-Insttene (self)
GPT-4.1
GPT-4.1 1112255-14B-Istrruee
20.0 (43.0)
Swen2.5-14B-Intruee
20.0 (+3.0)
Effectiveness Depends on Environment Complexity and Method Synergy The smaller gains for DG on simpler websites like "Shopping" suggest that when environment's dynamics align with common sense priors (c.g., clicking 'search' shows search bar), the explicit dynamics provide less new information, and the overhead of longer context may slightly hinder performance. In these simple settings, longer input on dynamics may negatively impact the model's effectiveness, as prior work has shown that excessively long contexts can hurt performance (Beltagy et al. [0200] Liu et al. 2024). We further analyze this phenomenon and provide exemplar environment dynamies in section A.5] Furthermore, our test of naive hybrid approach showed mixed results, underper- forming the dynamies grounding method alone BFCLv3 (21.0% vs 22.0%). This suggests simple combination is insufficient and may create interference between the two adaptation signals. principled integration of semantic guidance
A key challenge for future work is developing and distributional alignment.
4.3 ABLATIONS
Dynamics grounding enables self-improvement We ablate the choice of backbone LLMs for exploration and dynamics extraction in our dynamics grounding method (table Results show that using the same model for both components seeff-imrrvemment) performs as well as using a stronger model, indicating that our approach is robust the choice of exploration policy and dynamics extractor.
Effect of filtering environmental dynamics We conduct an ablation experiment BFCLv3 and find that filtering improves success rate across exploration budgets. Specifically, with 10 episodes, success rate rises from 61.0 to 64.0 (+3.0).
Syntactic alignment robust to different hyperparameters As shown in table training with different learning rates generally improves performance across range of reasonable hyperparameter values. However, using extreme hyperparameters can degrade performance. For example, in the case of the 7B model, applying very high learning rate of 1.0 training for 5 steps leads to reduced success rates, highlighting the importance of careful hyperparameter selection for optimal adaptation.
Larger models benefit from a larger learning rate When testing with a very large learning rate (i.e., learning rate of 1.0), we find that larger models (i.e., 14B and 32B models) benefit more with larger learning rate. We attribute this to the fact that the dimension of the adaptation vector for 14B and 32B models is the same (i.e., 5120), and is much larger compared the 7B model's (i.e., 3584).
5 LIMITATIONS AND FUTURE WORK
Our study is primarily focused on the Qwen2.5 model family for syntactic alignment; future work should validate these findings across a broader range of open-source architectures. Our dynamics grounding method currently operates in environments with explicit state transitions, where testbeds such user-oriented conversational tasks not explored. Additionally, our online syntactic align- ment method does not currently normalize updates by the hidden dimension size, which could im- prove robustness to hyperparameters.

---

Published as a conference paper at ICLR 2026
The most promising direction for future work is the development of principled method for integrat- ing our two adaptation strategies as our analysis suggests that simply combining them suboptimal. A more advanced approach could involve meta--ontrooler that assesses an environment's complex- ity to dynamically decide whether to rely on efficient online adaptation for simple tasks or to deploy the more costly dynamics grounding for complex, unfamiliar environments.
CONCLUSIONS
In this work, presented comparative study of two distinct strategies for adapting LLM agents to novel, complex environments. We identified two primary failure modes-syntactic and semantic mismatches- and investigated targeted solutions for each. The first, syntactic alignment (SA), is an efficient technique for aligning an agent's 's output distribution to the local format an environment. The second, dynamics grounding (DG), is an effective and efficient method for providing the agent with a causal world model through proactive dynamics extraction.
Our evaluations diverse agentic benchmarks demonstrate that both strategies significantly im- prove performance. Online adaptation provides consistent, low-cost gains, while dynamics ground- ing is critical for success in complex settings where an agent's pre-trained knowledge fails. Our findings highlight that multi-faceted approach to adaptation is necessary for robust agent gen- eralization and point toward exciting future directions in creating more intelligent and integrated adaptation systems.
ACKNOWLEDGMENTS
We thank Salesforce AI Research for supporting this work. We also thank the reviewers and the area chairs for their time and thoughtful feedback.
REFERENCES
Ekin Akyürek, Mehul Damani, Adam Zweiger, Linlu Qiu, Han Guo, Jyothish Pari, Yoon Kim, and Jacob Andreas. The Surprising Effectiveness of Test-Time Training for Few-Shot Learning, March 2025. htt.://pp///a..../////a/////////
07279 utXiv:2411.07270 [cs].
Iz Beltagy, Matthew E. Peters, and Arman Cohan. Longformer: The Long- Long-Document Transformer, December 2020. URL
arXiv:2004.05150 [cs].
Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wort- man Vaughan. A theory of learning from different domains. Machine Learning, 1911)151 May 2010. ISSN 1573-0565, doi: 10. 100100077 URL https:t/dci..ooo 10. 1ww11007/119944000--111100
Hyungjoo Chae, Namyoung Kim, Kai Tza-iunn Ong, Minju Gwak, Gwanwoo Song. Jihoon Kim, Sunghwan Kim, Dongha Lee, and Jinyoung Yeo. Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation, March 2025. URL http://arka
atq/abb/24100 13232 arXiv:2410.1 132232 [cs].
Zehui Chen, Kuikun Liu, Qiuchen Wang, Wenwei Zhang, Jiangning Liu, Dahua Lin, Kai Chen, and Feng Zhao. Agent-FLAN Designing Data and Methods of Effective Agent Tuning for Large Language Models. arXiv, 2024. doi: 00.48500AAXXI..443...222 12881. URL https::/
org/abs 2403. 12881 Version Number: 1.
Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang. Huan Sun, and Yu Su. Mind2Web: Towards Generalist Agent for the Web, December 2023. URL http: 06070 arXiv:2306.06070 [cs].
Jingtao Ding, Yunke Zhang, Yu Shang, Yuheng Zhang, Zefang Zong, Jie Feng, Yuan Yuan, Hongyuan Su, Nian Li, Nicholas Sukiennik. Fengli Xu, and Yong Li. Understanding World or Predicting Future? A Comprehensive Survey of World Models. ACM Computing Sur- veys, 58(3):1-88, February 2026. ISSN 0360-0300, 1557-7341. doi: 10.1145/3746449, URL https://d1 acm. www..oorg/ooooooo http.///////....11//om /ttp://do//0.111111//44444444
10

---

Published as a conference paper at ICLR 2026
Tianqing Fang, Hongming Zhang, Zhisong Zhang, Kaixin Ma, Wenhao Yu, Haitao Mi, and Dong Yu. WebEvolver: Enhancing Web Agent Improveepoee with Coevolving World Model. 2025, doi: 11448550/ARXXI22544.11000. arXiv Version Number: 2.
orq /bs///b///a/ 2504. 21024 Publisher:
Jay W. Forrester. Counterintuitive behavior of social systems. Theory and Decision, 2(2):109-140, December 1971. ISSN 1573-7187. doi: 10. 10.1007/BF00148991. URL https: /tttps://do..or/ 0. 101100000000899500
Boyu Gou, Zanming Huang, Yuting Ning, Yu Gu, Michael Lin, Weijian Qi. Andrei Kopanev, Botao Yu, Bemal Jiménez Gutiérrez, Yiheng Shu, Chan Hee Song. Jiaman Wu, Shijie Chen, Hanane Nour Moussa, Tianshu Zhang, Jian Xie, Yifei Li, Tianci Xue, Zeyi Liao, Kai Zhang. Boyuan Zheng, Zhaowei Cai, Viktor Rozgic, Morteza Ziyadi, Huan Sun, and Yu Su. Mind2Web 2: Evaluating Agentic Search with Auudee July 2025. URL URL httt:://arxiv.org urXiv:2506.21500 [cs].
abs/2506. 21506
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshom, Aobo Yang, Archi Mitra, Archic Sravankumar, Artem Ko- renev, Arthur Hinsvark. Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, Danny Wyatt, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Francisco Guzmán, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Govind That- tai, Graeme Nail, Gregoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Kore- vaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel Kloumann, Ishan Misra, Ivan Evtimov, Jack Zhang, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Ma- hadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jong- soo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Karthik Prasad, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield. Kevin Stone, Khalid El-Arini, Krithika Iyer, Kshitiz Malik, Kuenley Chiu, Kunal Bhalla, Kushal Lakhotia, Lauren Kantala-Yeary, Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Mannat Singh, Manohar Paluri, Marcin Kardas, Maria Tsimpoukelli, Mathew Oldham, Mathieu Rita, Maya Pavlova, Melanie Kambadur, Mike Lewis, Min Si, Mitesh Ku- mar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoy- chev, Niladri Chatterji. Ning Zhang, Olivier Duchenne, Onur Çelebi, Patrick Alrassy, Pengchuan Zhang, Pengwei Li, Petar Vasic, Peter Weng, Prajjwal Bhargava, Pratik Dubal, Praveen Krishnan, Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy, Ra- mon Calderer, Ricardo Silveira Cabral, Robert Stojnic, Roberta Railcanu, Rohan Maheswari, Ro- hit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang, Saghar Hosseini, Sahana Chennabasappa, Sanjay Singh, Sean Bell, Seohyun Sonia Kim, Sergey Edunov, Shaoliang Nie, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan, Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan, Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasha, Thomas Georgion, Thomas Scialom, Tobias Speckbacher, Todor Mi- haylov, Tong Xiao, Ujjwal Karn. Vedanuj Goswami. Vibhor Gupta. Vignesh Ramanathan, Viktor Kerkez, Vincent Gonguet, Virginie Do. Vish Vogeti, Vitor Albicro, Vladan Petrovic, Weiwei Chu, Wenhan Xiong, Wenyin Fu, Whitney Meers, Xavier Martinet, Xiaodong Wang, Xiaofang Wang, Xiaoqing Ellen Tan, Xide Xia, Xinfeng Xie, Xuchao Jia, Xuewei Wang, Yaelle Gold- schlag, Yashesh Gaur, Yasmine Babaei, Yi Wen, Yiwen Song. Yuchen Zhang, Yue Li, Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen, Zoe Papakipos, Aaditya Singh, Aayushi Srivastava, Abha Jain, Adam Kelsey, Adam Shajnfeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alexei Baevski, Allie Feinstein, Amanda Kallet, Amit Sangani, Amos Teo, Anam Yunus, Andrei Lupu, Andres Alvarado, An- drew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramchandani, An-
11

---

Published as a conference paper at ICLR 2026
nie Dong, Annie Franco, Anuj Goyal, Aparajita Saraf, Arkabandhu Chowdhury, Ashley Gabriel, Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leon- hardi, Bernie Huang. Beth Loyd, Beto De Paola, Bhargavi Paranjape, Bing Liu, Bo Wu, Boyu Ni, Braden Hancock, Bram Wasti, Brandon Spence, Brani Stojkovic, Brian Gamido, Britt Mon- talvo, Carl Parker, Carly Burton, Catalina Mejia, Ce Liu, Changhan Wang. Changkyu Kim, Chao Zhou, Chester Hu, Ching-Hsiang Chu, Chris Cai, Chris Tindal, Christoph Feichtenhofer, Cynthia Gao, Damon Civin, Dana Beaty, Daniel Kreymer, Daniel Li, David Adkins, David Xu, Davide Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang, Due Le, Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery, Eleonora Presani, Emily Hahn, Emily Wood, Eric-Tuan Le, Erik Brinkman, Esteban Arcaute, Evan Dunbar, Evan Smoth- ers, Fei Sun, Felix Kreuk, Feng Tian, Filippos Kokkinos, Firat Ozgenel, Francesco Caggioni, Frank Kanayet, Frank Seide, Gabriela Medina Florez, Gabriella Schwarz, Gada Badeer, Georgia Swee, Gil Halpern, Grant Herman, Grigory Sizov, Guangyi, Zhang, Guna Lakshminarayanan, Hakan Inan, Hamid Shojanazeri, Han Zou, Hannah Wang. Hanwen Zha, Haroun Habeeb, Harri- son Rudolph, Helen Suk, Henry Aspegren. Hunter Goldman, Hongyuan Zhan, Ibrahim Damlaj, Igor Molybog, Igor Tufanov, Ilias Leontiadis, Irina-Elena Veliche, Itai Gat, Jake Weissman, James Geboski, James Kohli, Janice Lam, Japhet Asher, Jean-Baptiste Gaya, Jeff Marcus, Jeff Tang, Jen- nifer Chan, Jenny Zhen, Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang. Joe Cummings, Jon Carvill. Jon Shepard, Jonathan McPhie, Jonathan Torres, Josh Ginsburg, Jun- jie Wang. Kai Wu, Kam Hou U, Karan Saxena, Kartikay Khandelwal, Katayoun Zand, Kathy Matosich, Kaushik Veeraraghavan, Kelly Michelena, Keqian Li, Kiran Jagadeesh. Kun Huang, Kunal Chawla, Kyle Huang, Lailin Chen, Lakshya Garg, Lavender A, Leandro Silva, Lee Bell, Lei Zhang, Liangpeng Guo, Licheng Yu, Liron Moshkovich, Luca Wehrstedt, Madian Khabsa, Manav Avalani, Manish Bhatt, Martynas Mankus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Keneally, Miao Liu, Michael L. Seltzer, Michal Valko, Michelle Restrepo, Mihir Patel, Mik Vyatskov, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang, Miquel Jubert Hermoso, Mo Metanat, Mohammad Rastegari, Munish Bansal, Nandhini Santhanam, Natascha Parks, Natasha White, Navyata Bawa, Nayan Singhal, Nick Egebo, Nicolas Usunier, Nikhil Mehta, Nikolay Pavlovich Laptev, Ning Dong. Norman Cheng, Oleg Chernoguz, Olivia Hart, Omkar Salpekar, Ozlem Kalinli, Parkin Kent, Parth Parckh, Paul Saab, Pavan Balaji. Pedro Rittner, Philip Bontrager, Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandani, Pritish Yuvraj. Qian Liang. Rachad Alao, Rachel Ro- driguez, Rafi Ayub, Raghotham Murthy, Raghu Nayani, Rahul Mitra, Rangaprabhu Parthasarathy, Raymond Li, Rebekkah Hogan, Robin Baitey, Rocky Wang, Russ Howes, Ruty Rinott, Sachin Mehta, Sachin Siby, Sai Jayesh Bondu, Samyak Datta, Sara Chugh, Sara Hunt, Sargun Dhillon, Sasha Sidorov, Satadru Pan, Saurabh Mahajan, Saurabh Verma, Seiji Yamamoto, Sharadh Ra- maswamy, Shaun Lindsay, Shaun Lindsay, Sheng Feng, Shenghao Lin, Shengxin Cindy Zha, Shishir Patil, Shiva Shankar, Shuqiang Zhang. Shuqiang Zhang. Sinong Wang, Sneha Agarwal, Soji Sajuyigbe, Soumith Chintala, Stephanie Max, Stephen Chen, Steve Kehoe, Steve Satter- field, Sudarshan Govindaprasad, Sumit Gupta, Summer Deng, Sungmin Cho,, Sunny Virk, Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Koehler, Thomas Robinson, Tianhe Li, Tianjun Zhang, Tim Matthews, Timothy Chou, Tzook Shaked, Varun Vontimitta, Victoria Ajayi, Victoria Montanez, Vijai Mohan, Vinay Satish Kumar, Vishal Mangla, Vlad Ionescu, Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov. Wei Li, Wenchen Wang, Wenwen Jiang, Wes Bouaziz, Will Constable, Xiaocheng Tang. Xiaojian Wu, Xiaolan Wang. Xilun Wu, Xinbo Gao. Yaniv Kleinman, Yanjun Chen, Ye Hu, Ye Jia, Ye Qi, Yenda Li, Yilin Zhang, Ying Zhang, Yossi Adi, Youngjin Nam, Yu, Wang, Yu Zhao, Yuchen Hao, Yundi Qian, Yunlu Li, Yuzi He, Zach Rait, Zachary De Zef Rosnbrick, Zhaoduo Wen, Zhenyu Yang, Zhiwei Zhao, and Zhiyu Ma. The Llama 3 Herd of Models, November 2024. URL fittp://arxxr org/ abs www./rrg//////.e/ee...eeeeeeee/e
21783 arXiv:2407.21788 [cs].
Izzeddin Gur, Hiroki Furuta, Austin Huang, Mustafa Safdari, Yutaka Matsuo, Douglas Eck, and Aleksandra Faust. A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis. 2023. doi: 00.4850//RRXV.23077 12856. URL htlttttst///xii// abs / 2307. 1 12856 Publisher: arXiv Version Number: 4.
David Ha and Jürgen Schmidhuber. World Models. March 2018. doi: 10.5281/zenodl / 1207631.
_ http::///wwwwwwwo/////////////// wwtp/rg/////.../// 10122 arXiv: 10122 [es).
12

---

Published as a conference paper at ICLR 2026
Hongliang He, Wenlin Yao, Kaixin Ma, Wenhao Yu, Yong Dai, Hongming Zhang, Zhenzhong Lan, and Dong Yu. WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models, June 2024. URL/attp://axxx wrg. abs htw.g//////.......ggg// 13919 1IXiv:2401.39119 [cs].
Yang Hu, Xingyu Zhang, Xueji Fang, Zhiyang Chen, Xiao Wang, Huatian Zhang, and Guojun Qi. SLOT: Sample-specific Language Model Optimization at Test-time, May 2025. URL http:
// //arxii..rg/abs//550000
12392 arXiv:2505. 12392 [es].
Tomoyuki Kagaya, Thong Jing Yuan, Yuxuan Lou, Jayashree Karlekar, Sugiri Pranata, Akira Ki- nose, Koki Oguri, Felix Wick, and Yang You. RAP: Retrieval-Augmentté Planning with Con- textual Memory for Multimodal LLM Agents. 2024. doi: 10. 4485500/AXX..242200000000
URL
https://aas /arxivv org/ abs 2402. 03610 Publisher: arXiv Version Number: 1.
Pangyu Lei, Jixuan Chen, Yuxiao Ye, Ruisheng Cao, Dongchan Shin, Hongjin Su, Zhaoqing Suo, Hongeheng Gao, Wenjing Hu, Pengcheng Yin, Victor Zhong, Caiming Xiong, Ruoxi Sun, Qian Liu, Sida Wang, and Tao Yu. Spider 2.0: Evaluating Language Models on Real-Workee Enterprise Text-to-SQL. Workflows, November 2024. URL http: http://axxxvv..o//bb.b/o hbwgg//bs/4447 .07663 1.0773341.007763 [cs].
Jinyang Li, Binyuan Hui, Ge Qu, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Rongyu Cao, Ruiying Geng, Nan Huo, Xuanhe Zhou, Chenhao Ma, Guoliang Li, Kevin C. C. Chang, Fei Huang, Reynold Cheng, and Yongbin Li. Can LLM Already Serve A Database Interface? A Blg Bench for Lcrge-Scale Database Grounded Text-to-SQLs, November 2023a.
URL http:://taxiiii org/ abs 03111 arXiv:2305.0311 [cs].
Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenherg. Inference-Time Intervention: Eliciting Truthful Answers from Language Model. Ad- vances in Neural Information Processing Systems, 36:41451-41530 December 2023b.
URL https: https://prroceedings www.:eeewwwiinnnnn.se...e....eee nwurips.r.c.. cct forrr_fillee wwwes/ieeeeeeeee 3ttpppppee...eeepppp/////////// hashh/ 01233300330732220044777866633333
Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the Middle: How Language Models Use Long
Contexts. Transactions of the Association for Computational Linguistics, 12:157- 173, February 2024. ISSN 2307-387X.
doi: 10.1162/tacLa.00033,, URL https:
//direct.min mit. wwtp://e//at..eeeeeeeeee
htt.:/e.......eeeeeeeaaaaaaaaa.d 11t2//tccc./ 00638/11963C/
Lost fost-in-thar Midee-Midaa adde Hww
Long
Linhao Luo, Yuan-Fang Li, Gholamreza Haffari, and Shirui Pan. Reasoning on Graphs: Faithful
and Interpretable Large Language Model Reasoning. 2023. doi: 1048500AR.1V33000000...
URL https:://// / /xwp///xxxiiia iv hbtrgg/abs/23111 01061 Publisher: arXiv Version Number: 2.
Xing Han Là, Zdeněk Kasner, and Siva Reddy. WebLINX: Real-World Website Navigation with Multi-Turn Dialogue, September 2024. URL
arXiv:2402.05930 [es].
Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. Cross-Task General- ization via Natural Language Crowdsourcing Instructions. In Proceedings of the 60th Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3470- 3487, Dublin, Ireland, 2022. Association for Computational Linguisties. doi: 10.18653/v1/2222 acl-long.244. / wotthhoooyyoorg/20000 wtw.:/22..1looon..... 244
Shikhar Murty, Hao Zhu, Dzmitry Bahdanau, and Christopher D. Manning- NNetNav: Unsuper- vised Learning of Browser Agents Through Environment Interaction in the Wild, February 2025.
URL http:http///iiii org/ abs/ 02907 arXiv:2410.02907 [cs].
Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, and Mingkui Tan. Efficient Test-Time Time Model Adaptation without Forgetting. arXiv, 2022. doi: 10.48550/ Version Number: 2.
ARXIV.2204.02110 WtRl.ttps://// http:s://aaavvvooo http///awwwvovogob//////aooo
026110 / 2ttps///2242.220
OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red
13

---

Published as a conference paper at ICLR 2026
Avila, Igor Babuschkin, Suchir Balaji. Valerie Balcom, Paul Baltescu, Haiming Bao, Moham- mad Bavarian, Jeff Belgum, Irwan Bello, Jake Berdine, Gabriel Germadett-Shapine Christopher Berner, Lenny Bogdonoff, Oleg Boiko, Madelaine Boyd, Anna-Luisa Brakman, Greg Brock- man, Tim Brooks, Miles Brundage, Kevin Button, Trevor Cai, Rosie Campbell, Andrew Cann, Brittany Carey, Chelsea Carlson, Rory Carmichael, Brooke Chan, Che Chang. Fotis Chantzis, Derek Chen, Sully Chen, Ruby Chen, Jason Chen, Mark Chen, Ben Chess, Chester Cho, Casey Chu, Hyung Won Chung, Dave Cummings, Jeremiah Currier, Yunxing Dai, Cory Decareaux, Thomas Degry, Noah Deutsch, Damien Deville, Arka Dhar, David Dohan, Steve Dowling, Sheila Dunning, Adrien Ecoffet, Atty Eleti, Tyna Eloundou, David Farhi, Liam Fedus. Niko Felix, Simón Posada Fishman, Juston Forte, Isabella Fulford, Leo Gao, Elie Georges, Christian Gib- son, Vik Goel, Tarun Gogineni, Gabriel Goh, Rapha Gontijo-Lopes, Jonathan Gordon, Morgan Grafstein, Scott Gray, Ryan Greene, Joshua Gross, Shixiang Shane Gu, Yufei Guo, Chris Hal- lacy, Jesse Han, Jeff Harris, Yuchen He, Mike Hcaton, Johannes Heidecke, Chris Hesse, Alan Hickey, Wade Hickey. Peter Hoeschele, Brandon Houghton, Kenny Hsu, Shengli Hu, Xin Hu, Joost Huizinga, Shantanu Jain, Shawn Jain, Joanne Jang, Angela Jiang, Roger Jiang, Haozhun Jin, Denny Jin, Shino Jomoto, Billie Jonn, Heewoo Jun, Tomer Kaftan, Lukasz Kaiser, Ali Ka- mali, Ingmar Kanitscheider, Nitish Shirish Keskar, Tabarak Khan, Logan Kilpatrick, Jong Wook Kim, Christina Kim, Yongjik Kim, Jan Hendrík Kirchner, Jamie Kiros, Matt Knight, Daniel Kokotajlo, Lukasz Kondraciuk, Andrew Kondrich, Aris Konstantinidis, Kyle Kosic, Gretchen Krueger, Vishal Kuo, Michael Lampe, Ikai Lan, Teddy Lee, Jan Leike, Jade Leung. Daniel Levy. Chak Ming Li, Rachel Lim, Molly Lin, Stephanie Lin, Mateusz Litwin, Theresa Lopez, Ryan Lowe, Patricia Lue, Anna Makanju, Kim Malfacini, Sam Manning, Todor Markov, Yaniv Markovski, Bianca Martin, Katie Mayer, Andrew Mayne, Bob McGrew, Scott Mayer McKinney, Christine McLeavey, Paul McMillan, Jake McNeil, David Medina, Aalok Mchta, Jacob Menick, Luke Metz, Andrey Mishchenko, Pamela Mishkin, Vinnie Monaco, Evan Morikawa, Daniel Mossing, Tong Mu, Mira Murati, Oleg Murk, David Mély, Ashvin Nair, Reiichiro Nakano, Ra- jeev Nayak, Arvind Neelakantan, Richard Ngo, Hyeonwoo Noh, Long Ouyang, Cullen O' Keeff, Jakub Pachocki, Alex Paino, Joe Palermo, Ashley Pantuliano, Giambattista Parascandolo, Joel Parish, Emy Parparita, Alex Passos, Mikhail Pavlov, Andrew Peng, Adam Perelman, Filipe de Avila Belbute Peres, Michael Petrov, Henrique Ponde de Oliveira Pinto, Michael, Pokorny, Michelle Pokrass, Vitchyr H. Pong, Tolly Powell, Alethea Power, Boris Power, Elizabeth Prochl, Raul Puri, Alec Radford, Jack Rae, Aditya Ramesh, Cameron Raymond, Francis Real, Kendra Rimbach, Carl Ross, Bob Rotsted, Henri Roussez, Nick Ryder, Mario Saltarelli, Ted Sanders, Shibani Santurkar, Girish Sastry, Heather Schmidt, David Schnurr, John Schulman, Daniel Sel- sam, Kyla Sheppard, Toki Sherbakov, Jessica Shieh, Sarah Shoker, Pranav Shyam, Szymon Sidor, Eric Sigler, Maddie Simens, Jordan Sitkin, Katarina Slama, Ian Sohl, Benjamin Sokolowsky, Yang Song, Natalie Staudacher, Felipe Petroski Such, Natalie Summers, Ilya Sutskever, Jie Tang, Nikolas Tezak, Madeleine B. Thompson, Phil Tillet, Amin Tootoonchian, Elizabeth Tseng, Pre- ston Tuggle, Nick Turley, Jerry Tworek, Juan Felipe Cerón Uribe, Andrea Vallone, Arun Vi- jayvergiya, Chelsea Voss, Carroll Wainwright, Justin Jay Wang, Alvin Wang. Ben Wang. Jonathan Ward, Jason Wei, C. J. Weinmann, Akila Welihinda, Peter Welinder, Jiayi Weng. Lilian Weng, Matt Wiethoff, Dave Willner, Clemens Winter, Samuel Wolrich, Hannah Wong, Lauren Work- man, Sherwin Wu, Jeff Wu, Michael Wu, Kai Xiao, Tao Xu, Sarah Yoo, Kevin Yu, Qiming Yuan, Wojeiech Zaremba, Rowan Zellers, Chong Zhang, Marvin Zhang, Shengjia Zhao, Tianhao Zheng, Juntang Zhuang. William Zhuk, and Barret Zoph. GPT-4 Technical Report, March 2024. URL http::/arxiv> org/ abs/ abs/23033 2300/33333000 08774 arXiv:2303.0872. [cs].
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray. John Schulman, Jacob Hilton, Fraser Kel- ton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback, March 2022. URL BtRLbttp::/// aawii.iv........ www.or/.abbb..o.. abs /2003.223322 C2155 ar arXiv:2203.021: [cs].
Nina Panickssery, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. Steering Llama via Contrastive Activation Addition, July 2024. URI. http:
2312. 06681 arXiv:231
[cs].
Shishir G Patil, Huanzhi Mao, Fanjia Yan, Charlie Cheng-Jie Ji, Vishnu Suresh, lon Stoica, and Joseph E Gonzalez. The Berkeley Function Calling Leaderboard (BFCL): From Tool Use to Agentic Evaluation of Large Language Models.
14

---

Published as a conference paper at ICLR 2026
Akshara Prabhakar, Zuxin Liu, Ming Zhu, Jianguo Zhang, Tulika Awalgaonkar, Shiyu Wang, Zhi- wei Liu, Haolin Chen, Thai Hoang, Juan Carlos Niebles, Shelby Heinecke, Weiran Yao, Huan Wang, Silvio Savarese, and Caiming Xiong. APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplay, July 2025. URL. http://arxiv.orrt 03601 arXiv:2504.03601 [es].
Shuofei Qiao, Runnan Fang, Ningyu Zhang, Yuqi Zhu, Xiang Chen, Shumin Deng, Yong Jiang, Pengjun Xie, Fei Huang, and Huajun Chen. Agent Planning with World Knowledge Model. 2024. doi: 00448500/RXXV:2440 14205. URLhttps::// httptt::/aa......t/t wwxxiv.orgie wwwp.v..ogg/ab///25.... Publisher: arXiv Version Number: 4.
Qwen, An Yang. Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang. Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men. Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yugiong Liu, Zeyu Cui. Zhenru Zhang, and Zihan Qiu. Qwen2.5 Technical Report, MrXiv:2412.15115 [cs].
January 2025. URL http::tttaa/atta www. abs 22t.//412..111111
Viacheslav Sinii, Alexey Gorbatovski, Artem Cherepanov, Boris Shaposhmikov, Nikita Balagansky, and Daniil Gavrilov. Steering L.M Reasoning Through Bias-Only Adaptation, October 2025. URL httpthttt///rivi...eee org. /200.. 18 18706) arXiv:2505.1 18706 [cs].
Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-Time Training with Self-Supervision for Generalization under Distribution Shifts. In Proceedings of the 37th International Conference on Machine Learning, pp. 0229-9248. PMLR, November 2020. URL https::/ppoceecingg//
mlr
html
ISSN: 2640-3498.
Daniel Tan, David Chanin, Aengus Lynch, Dimitrios Kanoulas, Brooks Paige, Adria Garriga- Alonso, and Robert Kirk. Analyzing the Generalization and Reliability of Steering Vectors, May 2025. (RLLhttpp//axxiii wwt.//rgg/aaa.aoo.ee..ooooe
abs ab// 12404 arXiv:2407.1244 122:24 [cs].
Lucas-Andrei Thil, Mirela Popa, and Gerasimos Spanakis. Navigating WebAI: Training Agents to Complete Web Tasks with Large Language Models and Reinforcement Learning. Proceed- ings of the 39th ACM/SIGAPP Symposium
Applied Computing, pp. 866-874, April 2024. doi: 10. 1054114//6000 3633990.. URLhttps:: //dl: a acm. wwtg//////////oo/ooo//////oooo// 1145/
3635903 Conference Name: SAC '24: 39th ACM/SIGAPP Symposium Applied Computing Place: Avila Spain Publisher: ACM.
ISBN: 9798400702433
Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J. Vazquez, Ulisse Mini, and Monte MacDiarmid. Steering Language Models With Activation Engineering, October 2024. abss/308. 10248 arXiv:2308. 10248 [es].
URL http:http////iii..otm
Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Damell. Tent: Fully Test-time Adaptation by Entropy Minimization, March 2021. URL /atp::/ http://axxvv.orro abs abss2206.. 10726 MxXi:20006.10725 [cs].
Junjie Wang, Mingyang Chen, Binbin Hu, Dan Yang, Ziqi Liu, Yue Shen, Peng Wei, Zhiqiang Zhang, Jinjie Gu, Jun Zhou, Jeff Z. Pan, Wen Zhang, and Huajun Chen. Learning to Plan for Marrival-AAumeeteer Large Language Models from Knowledge Graphs.
2024a. doi:
00.48550/ARXIV.24000 2...XI.2446..1228.....
2406. 14262 Publisher:
arXiv Version Number: 3.
Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, and Graham Neubig. Agent Workflow Memory, September 2024b. URL htt/:httt///rxia www.arwwwww....aroaaoa www//orgsss....ooooo...eo..ooeee 2409. 07429, arXiv:2409.07429 [cs].
Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao. Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese. Caiming Xiong, Victor Zhong, and Tao Yu. OSWorld: Benchmarking Multimodal Agents for Open-Endes Tasks in Real Computer Environments, May 2024. URL http:// arxiv. www.vvorg/bbb.o.. 2404 07972 0XXv92404.007972 [cs).
15

---

Published as a conference paper at ICLR 2026
Yiheng Xu, Dunjie Lu, Zhennan Shen, Junli Wang, Zekun Wang, Yuchen Mao, Caiming Xiong, and Tao Yu. AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorials, December 2024. httt:t////aaaa http/rrxiv..... http://...rrab////////e abs/24122 09605 arXiv:4412.09605 [cs].
Ke Yang. Jiateng Liu, John Wu, Chaoqi Yang, R. Fung, Sha Li, Zixuan Huang, Xu Cao, Xingyao Wang, Yiquan Wang, Heng Ji, and Chengxiang Zhai. LLM Is the Wizard, Then Code Is the Wand: A Survey How Code Empowers Large Language Models Serve Intelligent Agents. 2024. doi: 10. 10.48550/RRXI.2244 20024400111122 Publisher: arXiv Version Number: 2.
00812]
Ke Yang. Yao Liu, Sapana Chaudhary. Rasool Fakoor, Pratik Chaudhari, George Karypis, and Huzefa Rangwala. AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents, May 2025. URL http::://aaaiiv..rr//bb.seeee/ss 13822 arXiv:2410.1388 [cs].
Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. tttaSS-bench: A Benchmark for Tool-Agent- User Interaction in Real-Worde Domains, June 2024. URL http: MrXiv:2406.1204/ [cs].
org/abs 2406 12045
Yanzhe Zhang, Tao Yu, and Diyi Yang. Attacking Vision-Language Computer Agents via Pop-ups, 02391 arXiv:241..02391 [cs]
November 2024. URL
version:
Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. WebArena: A Realistic Web Environment for Building Autonomous Agents, April 2024a. AtLpa:///aaxxi..oo 2b07 13854 arXiv:2307. 13854 [es].
Siyu Zhou, Tianyi Zhou, Yijun Yang, Guodong Long, Deheng Ye, Jing Jiang, and Chengqi Zhang. WALL-E: World Alignment by Rule Learning Improves World Model-based LLM Agents. 2024b. doi: 10. 148550/ARXIV.241..0007
URL hhttps://axxi// wwwpxxiv........ooo abs/
2410.07488
Publisher: arXiv Version Number:
Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan. Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael Byun, Zifan Wang. Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zico Kolter, and Dan Hendrycks. Representation Engincering: A Top-Down Ap- proach to AI Transparency, March 2025. URL ltttp://axii..ot wwwpaa/wwww..rg/////..3.../ 01405 arXiv:2310.01405 [cs].
Yuxin Zuo, Kaiyan Zhang. Li Sheng, Shang Qu, Ganqu Cui, Xuekai Zhu, Haozhan Li, Yuchen Zhang, Xinwei Long, Ermo Hua, Biging Qi, Youbang Sun, Zhiyuan Ma. Lifan Yuan, Ning Ding, and Bowen Zhou. TTRL: Test- Time Reinforcement Learning, June 2025. URL http: 16084 arXiv:2504.16084 [cs].
//axxiv.rq/abbat
A APPENDIX
A.1 EXPERIMENT DETAILS
WebArena We follow the BrowserGym guide to setup WebArena enviormment. We AWS EC2 instance with a 3a. xlarge instance. For all exploration episodes of dynamics grounding, temperatue of 1.0. We use temperature of 0.0 for all task evaluation experiments.
BFCLv3 For BFCLv3, follow the official setur and consistently using temperature of 0.
web bebaarnnaRFAAM....
16

---

Published as a conference paper at ICLR 2026
Tau-bench We adopt the codebase form (Prabhakar et al. 2025 and use Best-of-N (N=5) strategy for LLM-simulated user agent, so each trajectory will scored by am LLM judge and the reponse that best adheres with user intent will be select, therefore drastically mitigating variance in average reward.
A.2 PROMPTS USED IN WEBARENA EXPERIMENTS
Listing 1: prompt used to synthesize personas for WebArena
You are expert creating realistic website. You are given the name of a website and some example outputs for the website, generate diverse list of $ personas} personas, each with a unique "persona" name and 1-2 sentence "description" of their typical behavior. interests, or motivations on that site.
personas given
Format your output as JSON array where each element is object with two fields:
"persona"; short descriptive name for the persona (e.g., Casual Browser", "Tech Enthusiast") "description brief description of how this persona typically uses the website
Now, generate list of personas in the specified JSON format. You are given the name of the website and some example outputs for the website.
Website: S S(website)
Example outputs: S(examples)
Listing 2: prompt used for persona-driven exploration in WebArena
You are autonomous intelligent agent tasked with navigating web browser. Your objective is to simulate a task that a person might perform, by interacting with the browser through the use of specific actions.
Here 's the information you'll Il have:
The current web page accessibility tree: This is a simplified representation of the webpage. providing key information. The current web page's URL: This is the page you" re currently navigating.
The open tabs: These are the tabs you have open.
The previous action: This the action you just performed. It may be helpful to track your progress.
Trajectory: This is a sequence of natural language descriptions of the agent's interaction with the web-browser.
Person Description: The description of a specific kind of person whose task you supposed to simulate.
Environment dynamics: Descriptions of how states transition.
The actions you can perform fall into several categories:
Page Operation Actions:
'click [id]': This action clicks on an element with a specific id on the webpage.
'type [id] [content] [press the content into the field with id. By default. the "Enter" key pressed after typing unless press enter after is set to 0.
Use this to type
17

---

Published as a conference paper at ICLR 2026
'hover [id] Hover over element with id. press [key.comb] Simulates the pressing of a key combination on the keyboard ge. Ctrl+v).
'scroll directtion=oon up] Scroll the page up or down.
Tab Management Actions:
"new_tab" Open new, emply browser tab "tab-focus [tab-index]"s Switch the browser's 's focus to specific tab using its index.
cclose_tab Close the currently active tab
URL Navigation Actions:
"goto [ Navigate to a specific URL.
go back : Navigate to the previously viewed page.
go forward : Navigate to the next page (if a previous 'go-back' performed).
action
Completion Action: sstop dodone "I Issue this action when you are done.
Homepage:
If you want visit other websites, check out the homepage at http :// htmepage.......m com. It has a list of websites you can visit.
To be successful, it is very important to follow the following rules:
1. You should only issue action that is valid given the current observation
2. You should only issue action at time.
3. You should follow the examples to reason step by step and then issue the next action.
4. Generate the action in the correct format. Start with "In summary, the next action will perform is" phrase. followed by action inside action will perform is "'click [1234] ....
For example. "In summary, the next
5. To generate an interesting task, make sure you issue atleast 4 actions before stopping More interesting tasks typically
involve more interactions with the browser.
6. You can issuc atmost 40 actions before stopping but feel free to output the stop action early if you want to stop exploring Don's generate anything after stop
Here are the list of environmental dynamies of this environment:
OBSERVATION:
[1744] link 'HP CB782A#ABA 640 Inkjet Fax Machine (Renewed)*
[1749] StaticText $2299.9
[1757] button 'Add to Cart
[1760] button *Add to Wish List
[1761] button 'Add to Compare"
URL:
http :// onestopmarket com. office
offfice-prdducee -products office -fficce-elccroniess
html
ENVIRONMENTAL DYNAMICS:
[ ('initial state 'Homepage of onestopmarket com, with the button Add to Cart visible action :: "clikk [1757] where the button is purchase '11 PREVIOUS ACTION: None Let's think step bye-ep. This page lists the information of HP Inkjet Fax Machine. which is the product identified in the objective. Its price is $279 49. have checked the
18

---

Published as a conference paper at ICLR 2026
environmental dynamics and see that stopping with this answer from this state has not been done before. so this action will contribute to exploration diversity. I think I have achieved the objective. In summary, the next action I will perform is
stopop (9279.49]
OBSERVATION:
[164] textbox 'Search" focused: True required: False
[171] button 'Go"
[174] link 'Find directions between two points'
[212] heading 'Search Results"
[216] button 'Close"
URL:: http :// openstreetmap.co
ENVIRONMENTAL DYNAMICS:
H iiiiiiialllsss state "aeenStreetMap homepage with search box and Go button visible action 'click [171] to submit a search '}, ['initial_state
'OpenStreetMap homepage with search box
focused "action" : 'type [164] [parks near NY] [1]']]
PREVIOUS ACTION: None
Let s think step-by-step. This page has search box whose ID is the nominatim rule of openstreetmap, I can search for restaurants near a location by typing "restaurants near" followed by the location. pressing Enter afterwards. Before proceeding I will check the environmental dynamics to see if similar action has already been performed from this state. see that there is no previous action where, starting from the current state someone searched for "restaurants near CMU" in the search box. This means my action will contribute to exploration diversity In summary, the
[164]. According
can submit my typing by
action I will perform is "'type [164) [ restaurantss near CMU) [1]*..
OBSERVATION:
$ (OBSERVATION)
Person Deseription:
$ (PERSON DESCRIPTION)
URL: $ {UUL)
PREVIOUS ACTION: S (PREVIOUS ACTION)
ENVIRONMENTAL DYNAMICS:
(ENVIRONMENTAL DYNAMICS)
Listing 3: prompt used to extract environment dynamics from observations
You are given the output of action taken by an autonomous intelligent agent navigating a web browser. Your objective is to produce description of the changes made to the state of the browser. Specifically, you need to produce the following elements:
1) Initial state: provide concise description of the browser's state before the action was taken (for example: "GitHub homepage" or "GitHub repository page").
2) Environmental dynamics: clearly describe the specific changes that occurred in the browser state as a result of the action (for example: "The search bar expanded and is able to consume input").
Here s the information
have:
** Initial state of the browser as an accessibility :ree:** This is simplified representation of the webpage, providing key information.
Final state of the browser:**
19

---

** be successful, it is very important to follow these rules:*s Explicitly think about the various features on the website and how the interaction with the website changed these features. Published as a conference paper at ICLR 2026
This is the accessibility tree representation after the agent took the action.
The action taken by the web agent:**
The agent can take actions that fall under the following categories (with descriptions):
**
Page Operation Actions;**
eelick [id] Clicks on element with specific id the webpage.
Types the content into the field with id. By default, the "Enter" key is pressed after typing unless "press enter after set to 'hover [id] Hovers over an element with id.
"type [id] [content] prress.enter.attereeee
'press [key.comb] Simulates pressing key combination on the keyboard (c.g., Ctrl+v).
"scroll [direction=down up] Scrolls the page up or down.
Tabab Management Actions:**
'new_tab" Opens a new, empty browser tab "tab_focus [tab-index]': Switches the browser's focus to specific tab using its index.
"cooe tab :: Closes the currently active tab.
+*URL Navigation Actions:**
"goto [url] Navigates to specific URL.
"go_back", : Navigates to the previously viewed page.
'go-forward :: Navigates to the next page (if a previous 'go-back action performed).
Completion Action:**
"stop [ [answer]* : Issue this action when you believe the task is complete. If the objective is to find based answer, provide the answer in the bracket. If you believe the task is impossible to complete, provide the answer as "N/A" in the bracket.
2. Provide the description of changes in one or two sentences.
3. If there is change, your description of the changes in state should be "no change".
4. Strictly follow the ISON format in your response: {" initial.state" <description of initial state >, environmentallyyamiiss <description of environmental change >
Here are some output examples for some random tasks: ("initial_state" "Google homepage with search bar visible". environmental.dynmmmss"""
"A modal search dialog opened up bringing up a focused search input with suggestions allowing the to search jump to various GitHub sections."
2. ("initial.state" "Wikipedia article page", environmental dynamies": " "The page scrolled down, revealing the "History" section.")
3. ["initial_state" " "Amazon product page", duvironmental.yynamiie ": "A confirmation message appeared indicating the item added the cart
20

---

Published as a conference paper at ICLR 2026
Listing 4: prompt used to remove repeatitive trivial environment dynamics in WebArena
You are given list of environmental dynamics of an environment collected by an autonomous web browsing agent. You are tasked to remove repetitive and trivial environmental dynamics. Each of the point consists of the following:
1) initial state: provide a concise description of the browser state before the action was taken (for example: "GitHub homepage" or "GitHub repository page").
2) action: action taken make the environmental change
2) patrommental.dynamen that occurred in the browser's state as a result of the action (for example: "The search bar expanded and is able to consume input").
clearly describe the specific changes
To be successful, it is important to follow these rules:
1. entries with trivial enviornmental dynamics, such "when I scroll down, the page will reveal more content" should be removed. Trivial environmental dynamics are those that do not reflect a meaningful or unique change in the environment, such simple scrolling. pagination,
toggling the visibility of already available content without introducing new information interface elements.
2. Remove entries where the environmental dynamics only describe the expansion or collapse of text (e.g.. "The full abstract expanded and is now visible"), unless the expansion reveals new interface elements or options that were not previously accessible.
3. Remove entries where the arrrommental.dynaiie only describe navigation to different section of the same page (e.g., "The page scrolled to the 'References' section "), unless this navigation results in substantial change in the available interface or content.
4. Keep entries where the environmental describe: Unexpected or spurious state change of executing an action at the initial state. Keep entries where the environmental dynamics indicate that performing an action did not result in any change to the browser state even though a change would normally be expected (for example, clicking button that should open menu but nothing happens).
The appearance of controls, filters, options in the interface.
substantially different view (e g., from search results to a detailed article page). The addition or removal of search fields. filters. other interactive elements.
Navigation to new page
Any change that enables new actions or reveals new types of information not previously accessible.
5. When in doubt, prefer to remove entries that are repetitive (e. g.. multiple entries describing the same type of abstract expansion) or that do not add information about the environment capabilities or state.
21

---

Published as a conference paper at ICLR 2026
Your output should be cleaned list of environmental dynamics, with only the non-trivial non-repetitive and meaningful changes retained. Please output in the original JSON format.
Listing 5: prompt used for evaluation in WebArena
You are an autonomous intelligent agent tasked with navigating a web browser. You will be given web-based tasks. These tasks will be accomplished through the use of specific actions you can issue.
Here s the information you have:
The user objective: This is the task you you're re trying to complete. The current web page accessibility tree: This is a simplified representation of the webpage, providing key information. The current web page's s URL: This is the page you're currently navigating.
The open tabs: These are the tabs you have open.
The previous actions: These It may be helpful to track your progress.
all the action you have performed.
The actions you can perform fall into several categories:
Page Operation Actions:
'click [id] This action clicks on an element with specific id on the webpage.
'type [id] [content] [press_enter_et = Use this type the content into the field with id. By default, the "Enter" key pressed after typing unless press.enter.after is set to 0.
'hover [ Hover over an element with id. press [key comb] Simulates the pressing of key combination on the keyboard (e.g., Ctrl+v).
'scroll
Scroll the page up down
Tab Management Actions:
"new_tab": Open new, empty browser tab 'tab_focus [tab_index] Switch the browser's focus to specific tab using its index.
'close tab Close the currently active tab
URL Navigation Actions:
"ooto [ Navigate to specific URL. back :: Navigate to the previously viewed page...
fgo_frrwrrd :: Navigate to the next page (if previous 'go_back' action was performed).
Completion Action:
ssop [answer] Issue this action when you believe the task is complete. If the objective is to find text answer, provide the answer in the bracket. If you believe the task is impossible to complete, provide the answer as "N/A" in the bracket.
Homepage:
If you want visit other websites. check out the homepage http // homepage com. has a list websites you can visit.
22

---

Published as a conference paper at ICLR 2026
To be successful, it is very important to follow the following rules:
1. You should only issue action that is valid given the current observation
2. You should only issue action at time.
3. You should follow the examples to reason step by step and then issue the next action.
4. You are strictly forbidden from issuing goto action to URL that is not on the homepage.
5. Generate the action in the correct format. Start by reasoning about the current situation. End with "In summary, the next action will perform is" phrase, followed by action inside For example. "Let's think step-by bep-bb-step.. Given the current state I need to click the like button which has id 1234. In summary, the next action will perform is ... click [1234] ...
6. Issue stop action when you think you have achieved the objective. Don't generate anything after stop
Here are some example outputs for some random tasks:
1. Let's think step-by-step. This page list the information of HP Inkeet Fax Machine, which is the product identified in the objective. Its price is $279 .49. think I have achieved the objective. will issue the stop action with the answer. In summary, the next action I will perform is "'stop [$279 .49]
2. Let think step-by-step. This page has search box whose ID is [164]. According to the nominatim rule of openstreetmap
search for the restaurants near". can submit my typing by pressing the Enter afterwards. In summary, the next action I will perform is type [164] [restaurants near CMU] [
location by "restaurants
Listing 6: prompt used for evaluation on WehArena with environment dynamics
You are an autonomous intelligent agent tasked with navigating a web browser. You will be given web-based tasks. These tasks will be accomplished through the use of specific actions you can issue.
Here 's the information you' have:
The user objective: This is the task you' re trying to complete. The current web page's accessibility tree: This is a simplified representation of the webpage, providing key information. The current web page's URL: This is the page you' 'e currently navigating.
The open tabs: These are the tabs you have open.
The previous actions: These are all the action you have performed. It may be helpful to track your progress.
The environmental dynamics: You will also be given the environmental dynamics of the environment. You can use the environmental dynamics to predict the next action. Each environmental dynamics entry describes state change in the environment and contains the following fields: "initial.state" The state of the environment before the action is taken
"action". ": The action performed by the agent.
23

---

Published as a conference paper at ICLR 2026
"environmentalldyaamiss A description of how the environment changes as a result the action.
The actions you can perform fall into several categories:
Page Operation Actions:
'click [id] This action clicks on an element with specific id on the webpage.
'type [id] [content] pf
Use this type the content into the field with id. By default, the "Enter" key pressed after typing unless press enter after is set 0.
'hover [ Hover over an element with id. press pey_ccmmb Simulates the pressing of key combination on the keyboard (e.g.. Ctrl+v).
scrolll [down/upl Scroll the page up down
Tab Management Actions:
"new_tab" Open new, empty browser tab "tab_focus [tab_index] Switch the browser's focus to specific tab using its index.
'close tab Close the currently active tab
URL Navigation Actions: goto [url] Navigate to specific URL.
"go_back : Navigate to the previously viewed page...
gooworrwrdd :: Navigate to the next page (if previous 'go_back" action was performed).
Completion Action:
stop [answer] Issue this action when you believe the task is complete. If the objective is to find a text- answer, provide the answer in the bracket. If you believe the task is impossible to complete, provide the answer as "N/A" in the bracket.
Homepage:
If you want visit other websites, check out the homepage http :// http///mmppeeee../ com. has a list websites you can visit.
To be successful, it is very important to follow the following rules:
1. You should only issue an action that is valid given the current observation
2. You should only issue one action at time
3. You should follow the examples to reason step by step and then issue the next action.
4. You are strictly forbidden from issuing goto action that is not on the homepage.
URL 5. Generate the action in the correct format. Start by reasoning about the current situation. End with "In summary, the next action will perform is" phrase followed by action inside For example, "Let think step step-by-step. Given the current state need to click on the like button which has id 1234. In summary, the next action will perform is "'click [1234]
6. When selecting actions consider the environmental dynamics what state changes will occur based on your knowledge of how the website behaves. Use this to avoid actions that would lead
24

---

Published as a conference paper at ICLR 2026
to undesired states to strategically choose actions that lead to desired outcomes.
7. Issue stop action when you think you have achieved the objective. Don't t generate anything after stop
Here are some example outputs for some random tasks:
1. Let's think step-by-step. This page lists the information of HP Inkjet Fax Machine, which is the product identified in the objective. Its price is $279.49. By considering the environmental dynamics, are needed to change the state and have achieved the objective. I will issue the stop action with the answer. In the next action will perform is "sstop [$279 .49
can confirm that no further actions
2. Let s think step-by-step. This page has search box whose ID the nominatim rule of openstreetmap, I
is [164]. According can search for the restaurants near a location by "restaurants near". By considering the environmental dynamics, know that typing this query and pressing Enter will trigger the search and update the results accordingly. Therefore can submit my typing by pressing the Enter afterwards. In summary, the next action will perform is ''type [164] [restaurants near CMU) [1]*..
25

---

Published as a conference paper at ICLR 2026
A.3 PROMPTS USED BFCLV3 EXPERIMENTS
Listing 7: prompt used to synthesize exploration tasks on BFCLv3
You are expert at creating exploration strategies that will help guide an LLM to explore and understand a new function calling environment. Your task is synthesize some exploration goals based on a list of functions and a specific environment.
## Task
Given a list of functions and an environment, create S(N} distinct goals that will explore diffferent aspects of the environment Each goal should represent different exploration strategies and approaches to discovering functionality.
Input Format
** Enviroooemmn:::: The computing environment system (e.g., file system", " "vehiele control". "web browser") ** Functionns:: A list of available functions /commands alongside with detailed descriptions (e.g., ["cd()", "Is(", "pwd mkdir "I)()")
## Output Format Output a ISON list of goals. Each goal should be a single sentence that captures the exploration goal and the strategy to achieve it.
Format: ("goal description", "goal description", ...]
## Example Output
** E-Eiiromment**: File System
Functions
("name": "touch", "description "This tool belongs to the Gorilla file system. It is simple file system that allows users to perform basic file operations such as navigating directories, creating files and directories, reading and writing etc. Tool description: Create a new file of any extension in the current directory..
files
"parameters". ("type" "dict". properties": "rfilennam"" {"type": "string" "description" ": The name of the new file in the current directory. file name is local to the current directory and does not allow path ")). "required"; ['filc-name "]}, "response"; {"type" "dict",
properties": (}}}
("name": "we", "description" ": "This tool belongs to the Gorilla file system. It is simple file system that allows users to perform basic file operations such as navigating directories creating files and directories. reading and writing to files etc. Tool description: Count the number of lines, words, and characters in a file of any extension from current directory "parameters" ": ("type": " "ditt". "properties "; file name ": ("type" "string"
"dserippiion" "Name of the file of current directory to perform we operation on."). "mode": {" type": "string" "description" "Mode of operation ('1' for lines. 'w" for words. 'c' for characters). "H. "required":
"default"; " name "I]. "response", ["type", " "dict", "properties". ("count": f"type" "integer", "description": The count of the number of lines words, or characters in the file.") "type": ("type": "string", "description" "The type
26

---

## Your Task Published as a conference paper at ICLR 2026
of unit are counting. [Enum]] Iinlines \", ("words \", " "|"}|}
characters
("name": "Is". "description" " "This tool belongs to the Gorilla file system. It is a simple file system that allows users to perform basic file operations such navigating directories creating files and directories reading and writing to files etc. Tool description: List the contents of the current directory. "parameters"; {"type": "dict". "properties" {"a ": {"type": "boolean", "description" " "Show hidden files and directories. Defaults to False "default". false)) required"; []], "response". ("type": "dict", "properties"; (" cerrent_directory.com
("type". "array", "description" ": A list of the contents of the specified directory." "items". ("type": "]]ri
("name": "rm" "description" "This tool belongs to the Gorilla file system. It is simple file system that allows users to perform basic file operations such as navigating directories creating files and directories reading and writing etc. Tool description Remove a file or directory. parameters"; ("type": "dict", "properties". type": "string", "description" "The name of the file or directory to remove. "]). " "required"; f' name"]] response", ("type": "dict", "properties": f"result"; ("type": "string", "description" : "The result of the remove operation ."}}}
files.
("file-name" {"
Generated goals:
"Explore simple ways to create a file with touch (), count words of the file with we(), and remove the file with mm)." "Understand the "file "file_name" argument of the (ouch function to see if a file can be created using path instead of just file name."
Guidelines
**Be (unction-specifii which specific functions/ tools will be used and what sequence
Each goal should explicitly mention
** Incorporate function descriptions into goal formulation #*: You should examine the given function descriptions (i.e., input or output arguments) to propose goals to solve confusing or ambiguous function descriptions or environmental dynamics. ** =*Eure clarity and achievability *x; Each goal should be single, clear action that can be completed with the available functions
Now synthesize goal descriptions for the environment and functions provided in the format of JSON list. Please do not leave any comment.
Listing 8: user prompt used in BFCLv3 exploration
Do you think you have well-explored all functions that you are interested in particularly those with ambiguous input formats strrctuurss or potentially unexpected outputs?
27

---

Published as a conference paper at ICLR 2026
Consider stopping (###STOP) if:
You have thoroughly tested functions with unclear parameter requirements
You have explored functions that might produce unexpected or
variable output formats
You have sufficient understanding of how to handle edge cases and errors
Continue exploring (###CONTINUE) if:
There are still functions with ambiguous input specifications that need clarification
You haven 't fully tested functions that could have unpredictable outputs
You need more exploration to understand parameter constraints and error handling
Please output ###STOP if you believe you have comprehensive coverage, or ###CONTINUE if more exploration is needed.
Listing 9: prompt used extract environment dynamics from interaction logs of BFCLv3
You are an expert data analyst specializing in function calling environments. Your task is to analyze interaction logs between function calling bot and environment to extract environmental dynamics based on given exploration goal.
## Task
Given an exploration goal and an interaction log. extract environmental dynamics that describe how the environment changes in response to bot actions. Each environmental dynamic should capture the cadse-an-eeffect relationship between actions and environmental changes.
## Environmental Dynamic Structure Each environmental dynamic must include these three components:
1. ** the action Include context like current directory, variable values, file
The relevant state of the environment before
states, or any pertinent environmental conditions
-B specific about what matters for understanding the action 's impact
2. ** **action taken The exact function call action performed by the bot Include the function and all parameters with their values
UUse the exact syntax from the interaction log
3. cenviroommmtallyynnmissss How the environment changes as result of the action
Describe what was created. modified, deleted. or revealed Include any state transitions. data changes, or new information made available
Be specific about the observable effects
## Guidelines
** Output format**: Generate your response as a JSON array of objects
28

---

Published as a conference paper at ICLR 2026
Completeness Extract dynamics for every significant action in the log A*.cuuracy **: Base descriptions strictly what's observable in the interaction log
C*CCaarty **: Use precise, unambiguous language for each component
#*Focus**: Only include dynamics that show meaningful environmental changes
## Example Input Goal: Investigate how to create a new file with touch () and confirm its existence using ().
Interaction Logs:
Step 1:
Assistant: ()
Environment: {" curreettddreetoryy
[]]
Step 2:
Assistant: [mkdir
"tstdirrr)
Environment: None
Step 3:
Assistant: (e(foleer""eesddin
Environment:
"testdir")
Step 4:
Assistant: [pwd()
Environment: ["ourrent.worringgdrrccoooo "/// "///derrr
## Example Output
"initial_state "The current directory contains no files.". "action_take" pngirommentalddynamiii "A new file named \"testfile ixt is created in the current directory."
(xt
" "initial_state ": "The current directory contains the file \" testfile txt "
"action taken" ; "Is(),, "environmental dynamies": "The environment reveals the contents of the current directory. showing that \"testfile lxx) exists."
Listing 10: prompt used to filter environment dynamics BFCLv3
You are an expert data analyst specializing in function calling environments. Your task is to filter environmental dynamics remove insignificant or environment-speciffe details while preserving meaningful behavioral patterns.
## Task
Given ISON array of environmental dynamics, identify and remove dynamics that are:
1. ** Very environment-spccffice **: Contain specific data names, values that are unique to a particular environment instance/ setup
29

---

Published as a conference paper at ICLR 2026
2. ** Insignnificant Describe trivial non-meaningful changes that don't contribute to understanding function behavior (e.g
ls in file system)
3. .*Redundant Duplicate similar patterns already captured in other dynamies
## Filtering Criteria
### Remove dynamies that contain:
** Specific data values Names, IDs, file contents, user lists or any concrete data that varies between environments ** ==Fnvironnent-specffii paths*# Absolute paths, specific directory names, or location depeedent information identifiers **: User names, session IDs, timestamps, or other unique identifiers
** =#Instance-speee
**Tvival state changes**: Minor formatting changes, temporary states, or cosmetic updates
R*dunndant information*** Multiple dynamics describing the same behavioral pattern with different specific values
### Keep dynamics that describe:
General behavioral patterns*x: How functions typically behave
across different environments
Error conditions Standard error messages and failure modes ** State transitions General state changes (logged intou, file created/deleted, etc.)
** Function capabilities What functions
do, not what
specific data they return
Functioon contains meaningful arguments Keep dynamies where the function call includes arguments that affect behavior or state as these illustrate how input parameters influence outcomes.
## Guidelines
** Output format Return ISON array containing only the filtered dynamics
** Preserve structure Keep the same ISON structure for remaining dynamics
** Generalize descriptions Replace specific values with general descriptions where possible
Maintain accuracy *s: Don't change the core meaning of the dynamics, only remove specificity
Focuss on patterns =s: Prioritize dynamics that show reusable behavioral patterns
## Examples
### Remove (Very environment-specffi that contains data entries specific to environment instance): "jsson
"initial_state "No user is logged into the system, and a list of users needs to be identified." "action taken". "list () pnviroomentaalddyammiss
"Provides the list of available users: Alice, Bob, Catherine, and Daniel." "
30

---

Published as a conference paper at ICLR 2026
### Keep (General pattern and error message): json
"initial state": "No user is currently logged in.". "search. seearch_eessaee.. (keyword=(""est {)) aaniioommentlldynamiie "An error message is returned indicating that no user is currently logged in."
### Remove (Trivial transition): "'json
"initial state ": "In directory with existing files.", "action paction_taken" ; "Is(),
"environmental.yynnm
"Returns list of files and directories in the current location."
## Instructions
Analyze each environmental dynamic in the input ISON array
2. Apply the filtering criteria to determine if it should be kept or removed
3. For dynamics that are kept, generalize any remaining specific details
4. Return the filtered JSON array with only the significant generalizable dynamics
A.4 ADDITIONAL RESULTS
Table 6: Success rate (SR) of syntactic alignment across different Qwen2.5 instruct model sizes (7B, 14B, 32B) and training hyperparameters. Results are shown for varying numbers of training iterations and two learning rates (LR=0.1 and LR=1.0). Syntactic alignment adaptation generally improves SR over the baseline, with larger models and moderate learning rates yielding higher gains.
Model
Number of Training Iterations
SR (LR=0.1)
SR (LR=1.0)
Owen2.5-7BBInnttnu
Baseline (0)
9.00
Qwen2. Dwen2.5-7B-Instrucc
9,50
9.50
Qwen2.5-7B-Instruul
2
9.50
8.50
Qwen2.5-7BBIInsrruce
3
9.50
9,00
Dwen2.5-7B-Insttuu
4
10.00
8.50
Qwen2.5-7B-Insttee
5
10.00
8.50
Owen2.5-14B-Instrue
Baseline (0)
18.5
Owen2.5-14B-Innee
20.00
20.00
0wen2.5-14BB-Instnne
2
20.00
20.00
Owen2.5-14B-Inee
3
20.00
21.00
Qwenn.55-14B-Inttruet
4
19.50
20.00
Qwen2.55114B-Instrueet
5
19.00
19.00
Qwwen..5-32B-Inssrrucct
Baseline (0)
26.00
Qwenn2.5-32B--Instrna
25.50
26.50
Owen2.5-32B-Insee
2
26.50
26.50
Dwen2.5-32B-Innthe
3
26.50
27.00
Bwen2.5-33B-IInttee
4
26.50
27.00
Breen.5-33B-Inntie
5
26.00
26.50
31

---

Published as a conference paper at ICLR 2026
Table 7: Number of environment dynamics per website before and after filtering with GPT-4. 4. as the exploration policy and dynamics extractor. Most environment dynamics are filtered out.
Website
Number of Env Dynamics
Number of Env Dynamics (after filtering)
Shopping
334
52
Shopping Admin
334
50
GitLab
334
53
Map
334
18
Reddit
334
23
Table 8: Number of functions available in each BFCLv3 environment.
Environment
Num of Functions
VchicleControlAPI
22
GorillaFileSystem
18
TravelAPI
17
MathAPI
17
TwitterAPI
14
MessageAPI
10
TicketAPI
9
TradingBot
22
A.5 ENVIRONMENT DYNAMICS ANALYSIS
Here we provide some concrete examples of environment dynamics that are removed by our filtering mechanism. Exploration policy and dynamics extractor GPT-4.
No Change Dynamies
"initial_stated "Magento Admin product edit page, with the Country of Manufacture' dropdown expanded and no country selected.",
"action": "click (1405] where [1405] United states", aanviirommentll_yynnii.cc "no change"
Basic Dropdown/Combooox Expansion
"initial_state" "Magento Admin procuct edit page for 'sprite Stasis Ball 65 em with the 'Country of Manufacture' combodox (nctice-WPNSUSL) collapsed and not focused.", Maction": "click [1168] where [1168] is notice-wPNsJ5in, Menvire troreental_dynamicst
"The 'Country of Manufacture' combobox became focused and expanded, showing its list of country options."
Basic Dialog Open/Close
Minitial_state" "Magento Admin product edit page for 'sprite Stasis Ball 65 cm', with the 'ddd Attribute' button visible but attribute selection dialog open.",
"action": "click [720] where [720] is Add Attribute", annvirromenttl__ynnniisst "Clicking the 'Add Attribute' button
opened the 'Add Attribute' dialog, displaying available product attributes for selection and actions such 'Create New Attribute'."
32

---

Published as a conference paper at ICLR 2026
Standard Form Field Visibility
"initial_state", "Magentio Admin product detail page for 'sprite Stasis Ball 65 cm' with all product attribute forms visible, including collapsed 'Country of Kanufacture' dropdown combobox) ."
"action": "elick [1267] where [1267] notice-BRW2147", Senvirommontal__dnnnnle.. "The 'Country of Manufacture' combobox was clicked, causing attribute fields for 'Country of Manufacture' and all subsequent grouped attributes to become visible."
Basic Button Focus
"initial_state" "Magento Admin proauct edit page with the 'Add Attribute' dialeg open; in the attributes table, the 'Options column header includes checkbex and an rOptions' button, neither fecused nor expanded.",
"action": "click [2372] where [2372] httpr/orroeenaa___aaiiiiieee
"The 'Opticns' button in the 'Options' column header of the Add Attribute dialog becane focused and expanded drepdown list with single option, Seeecc All', now visible below button."
33


Reviews:
[{"review_id": "L9FCFMeJKX", "reviewer": "Reviewer_xmMr", "summary": "The paper proposes two method for test-time adaptation of LLM-based agents: a \"parametric\" and a \"non-parametric\" adaptation. The parametric adaptation proposes, for each episode, to learn a single adaptation vector which is added to the final hidden representation before the final projection layer, i.e., the vector acts as a logit bias. The non-parametric adaptation proposes a one-time deployment pipeline per environment: (1) derive exploration goals from the environment description via LLM calls, (2) run an LLM agent to explore and log (observation, action, new-observation) transitions, (3) have an LLM summarize these logs, (4) have a reasoning model filter out duplicate rules; at test time, the filtered rules are appended to the agent's context.", "questions": "I strongly suggest to contextualize the design choices for both the “parametric” and “non-parametric” adaptation methods in relation to prior work, highlighting key precedents and  differences.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "### Clarity\n\nThe writing is clear and free of typos. The figures are well designed and effectively illustrate the proposed adaptation strategies. The paper is also well structured: it opens by partitioning LLM-agent failures into two categories and then develops a corresponding adaptation strategy for each, yielding an easy-to-follow narrative.\n\n### Originality\n\nBased on my understanding of the literature, the proposed ideas seem reasonably novel in the context of LLM-based agents.\n\n### Quality\n\nThis paper (i) characterizes failure cases of LLM-based agents, (ii) formalizes deployment constraints that bound viable solutions, (iii) derives methods that adhere to those constraints, and (iv) evaluates them against established agent baselines.\n\n\n### Significance\n\nThe paper tackles out-of-distribution (OOD) performance in LLM-based agents which is an essential issue for robust deployment.", "weaknesses": "While Section 2 offers useful background on test-time adaptation and LLM-based agents, the paper stops short of situating its specific design choices within the existing literature. For instance, the proposed “parametric adaptation” appears to extend the methodology of “Steering LLM Reasoning Through Bias-Only Adaptation” [1] to the context of LLM-based agents.\n\n[1] Sinii, V., Gorbatovski, A., Cherepanov, A., Shaposhnikov, B., Balagansky, N., & Gavrilov, D. (2025, May 24). Steering LLM reasoning through bias-only adaptation (arXiv:2505.18706 [cs.LG], v1). arXiv. https://arxiv.org/abs/2505.18706", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CjvHRHXH6j", "reviewer": "Reviewer_uv4G", "summary": "This paper addresses the core challenge of Large Language Model (LLM) agents' poor generalization when facing novel, unseen environments. The authors innovatively decompose this challenge into two primary failure modes: Syntactic Mismatch and Semantic Mismatch.\n\nTo address these issues, the paper proposes two annotation-free Test-Time Adaptation (TTA) strategies:\n\n1. A parametric (PA) method that learns a lightweight adaptation vector, δ, to bias the model's output distribution, enabling rapid alignment with the environment's syntactic structure.\n2. A non-parametric (NPA) method that, prior to task execution, actively learns the environment's causal dynamics through an exploration phase and provides them as context to the agent, thereby constructing a non-parametric world model.", "questions": "1. Have the authors considered validating the effectiveness of both methods on a broader range of models, such as Llama-3 or GLM-4.5?\n2. For the non-parametric adaptation method, the comprehensiveness of the exploration (e.g., exploring the full functionality of a website, not just a subset) is critical. How do the authors ensure that the exploration is sufficiently comprehensive?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Introduces Test-Time Adaptation (TTA) into the domain of LLM agents, proposing targeted solutions for syntactic and semantic mismatches.\n2. The proposed methods are plug-and-play and exhibit strong adaptability.\n3. Provides detailed analysis and comprehensive ablation studies.", "weaknesses": "1. The combination of the parametric (PA) and non-parametric (NPA) methods does not demonstrate synergistic effects; in fact, their integration fails to yield better performance.\n2. The generalizability of the parametric adaptation method is not sufficiently demonstrated, as its experiments were confined to the Qwen-2.5 family of models.\n3. In more diverse and complex environments, the parametric adaptation method might require more training steps, leading to increased computational latency.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TzcGyXoStz", "reviewer": "Reviewer_oK2P", "summary": "This paper proposes 2 test-time adaptation methods for LLM-based agents, which allows the agent explore the environment before the task and update its parameter or context based on the exploration. The experiment shows that this method can improve the performance of the web agent compared with WAM (using world model to predict the next observation and filter the action)", "questions": "1. Explain each variation clearly, like I1:n−1 in equation 3.\n\n2. Fully polish your writing, make it more clear and easy to understand.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 3, "strengths": "1. This idea is natural but effective. The most prior knowledge in the environment agent have, the better the agent can perform.\n\n2. Lifelong learning is also a important topic. This paper try to give a solution to the problem by training. (Although its solution has some weaknesses)", "weaknesses": "1. The writing is really poor. Those 2 figures can't really present the idea of 2 methods.\n\n2. The classification of Syntactic Mismatch and Semantic Mismatch is meaningless. They are both the result of the lack of prior knowledge in the environment. Although the author try to seperate them by using 2 different methods, it looks like more weird, since they are both solve the lack of prior knowledge, no matter what prior knowledge it is. Such mapping is useless, it just like to increase the complexity of the paper.\n\n3. Almost half of the results in Table 2 is empty, which may cause unfair comparison. For example, is AWM really worse than your method in other task? (gpt-4o-mini & Qwen). Is it because the AWM's result comes from a old gpt-4o-mini model? You need to reapplied this method in another benchmarks to prove your claim.\n\n4. If the environment can't be explored, this method will not work, although I agree most of the environment can be explored.\n\nAs a result, although I quite like this idea, the soundness and the writing are not good enough, so I'll give a borderline score to this paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)