AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents
Authors: Ke Yang
Yao Liu
Sapana Chaudhary
Rasool Fakoor
Pratik Chaudhari
George Karypis
Huzefa Rangwala
Venue: ICLR 2025 Poster (2025)
Keywords: LLM
Agent
LLM-based Agent
Web Agent
Web Navigation
Primary area: foundation or frontier models, including LLMs
TL;DR: Our method, with new SOTA results on WebArena, demonstrates a strong zero-shot performance on web tasks and sheds light on the importance of observation and action space choice for LLM-based agents.

Abstract: Autonomy via agents based on large language models (LLMs) that can carry out personalized yet standardized tasks presents a significant opportunity to drive human efficiency. There is an emerging need and interest in automating web tasks  (e.g., booking a hotel for a given date within a budget). Being a practical use case itself, the web agent also serves as an important proof-of-concept example for various agent grounding scenarios, with its success promising advancements in many future applications. Meanwhile, much prior research focuses on handcrafting their web agent strategies (e.g., agent's prompting templates, reflective workflow, role-play and multi-agent systems, search or sampling methods, etc.) and the corresponding in-context examples. However, these custom strategies often struggle with generalizability across all potential real-world applications. On the other hand, there has been limited study on the misalignment between a web agent's observation and action representation, and the data on which the agent's underlying LLM has been pre-trained. This discrepancy is especially notable when LLMs are primarily trained for language completion rather than tasks involving embodied navigation actions and symbolic web elements. In our study, we enhance an LLM-based web agent by simply refining its observation and action space, aligning these more closely with the LLM's capabilities. This approach enables our base agent to significantly outperform previous methods on a wide variety of web tasks. Specifically, on WebArena, a benchmark featuring general-purpose web interaction tasks, our agent AgentOccam surpasses the previous state-of-the-art and concurrent work by 9.8 (+29.4%) and 5.9 (+15.8%) absolute points respectively, and boosts the success rate by 26.6 points (+161%) over similar plain web agents with its observation and action space alignment. Furthermore, on WebVoyager benchmark comprising tasks defined on real-world websites, AgentOccam exceeds the former best agent by 2.4 points (+4.6%) on tasks with deterministic answers. We achieve this without using in-context examples, new agent roles, online feedback or search strategies. AgentOccam's simple design highlights LLMs' impressive zero-shot performance on web tasks, and underlines the critical role of carefully tuning observation and action spaces for LLM-based agents.

Published as a conference paper at ICLR 2025
AGENTOCCAM: A SIMPLE YET STRONG BASELINE FOR LLM-BASED WEB AGENTS
Ke Yang Yao Liu Sapana Chaudhary Rasool Fakoor Pratik Chaudhari George Karypis Huzefa Rangwala University of Illinois Urbana- Chamma-gamm,inn, A key48i linois.edei {yaol iuai, chausapa, fakoor,
ABSTRACT
Autonomy via agents based on large language models (LLMs) that can carry out personalized yet standardized tasks presents significant opportunity to drive hu- man efficiency. There is an emerging need and interest in automating web tasks (e.g., booking hotel for given date within budget). Being a practical case itself, the web agent also serves as an important proof-of-- cooffoff-concept example for various agent grounding scenarios, with its success promising advancements in many future applications. Mcanwhile, much prior research focuses on handcraft- ing their web agent strategies (e.g.. agent's prompting templates, reflective work- flow, role-play and multi-agent systems, search or sampling methods, etc.) and the corresponding in-context examples. However, these custom strategies often strug- gle with generalizability across all potential real-world applications. On the other hand, there has been limited study on the misalignment between web agent's s observation and action representation, and the data on which the agent's underly- ing LLM has been pre-trained. This discrepancy is especially notable when LLMs are primarily trained for language completion rather than tasks involving embod- ied navigation actions and symbolic web elements. In our study, we enhance an LLM-based web agent by simply refining its observation and action space, align- ing these more closely with the LLM's capabilities. This approach enables our base agent significantly outperform previous methods on a wide variety of web tasks. Specifically. on WebArena, a benchmark featuring veneral-purpose. web in- teraction tasks, our agent AGENTOCCAM surpasses the previous state-of-the-ard and concurrent work by 9.8 (+29.4%) and 5.9 (+15.8%) absolute points respec- tively, and boosts the success rate by 26.6 points (+ (++661% %) over similar plain web Web
agents with its observation and action space alignment. Furthermore, ager benchmark comprising tasks defined on real-world websites, AGENTOCCAM exceeds the former best agent by 2.4 points (+4.6%) tasks with deterministic answers. We achieve this without using in-context examples, new agent roles, online feedback or search strategies. AGENTOCCAM'S simple design highlights LLMS' impressive zero-shot performance on web tasks, and underlines the critical role of carefully tuning observation and action spaces for LLM- agents."
INTRODUCTION
Al agents leveraging large language models (LLMs) show great potential in automating repetitive and programmatic tasks and thereby alleviating human workloads (Gao et al., 2024; Xi et al., 2023; Yang al., 2024). LLMs showcase remarkable capabilities in perception, reasoning and planning primarily due to their pre-training and post-learning. However, their effectiveness is significantly constrained when task-specific observation and action representations diverge from the parametric knowledge encoded during their training/learning time. For instance, in web-based tasks, these agents perform notably below human levels (Zhou et al., 2023b; Koh et al., 2024a).
To improve web task performance by -ased agents, recent work focuses on designing better agent policies with either handcrafted prompting templates (Sodhi et al., 2024) or hard-coded auto-
Work performed while interning at Amazon.
Our code and data available at http://tthllccccmmmmooooccoc//et

---

Published as a conference paper at ICLR 2025
Aligned
Original
My goal is, to list neviewers, ony, who mentiion cups being small.
Actions
click
type
nota
click
stop
back
branch
tab
back
action A
Given the web observation and available actions, click "neview" linld
Action Space: Compact
Àgen Ogcamt Action and Observation Space Alignment
Observation Space: Less Redundant Yot As Informative
Server
Previous Approaches: Compound LLM Policy
Original Observation
LLM Pollcy
(oodihuafere (1) Weshoa Rondphance J-anest Trea)
(1912 My Acpooit
My Accourr
Agent
observation Og
Rostrhesskee Wirebeee Heaetphored Inc (1312) Myy Aussunt
Aligned Observation
knc (1anrt My winn lid Riowed
Staticlod (18)) berra...
liv (1310) My Wish List lemo
Environment
Figure 1: Overview of AGENTOCCAM. Unlike prior research that works intensively on designing compound LLM policies, we enhance web agent simply by aligning the web interaction action and observation space with the functioning LLM's acquired knowledge and skills during its training.
prompting strategies (Fu et al., 2024; Wang et al., 2024). While those pre-defined strategies can be effective for certain tasks, they struggle to generalize to diverse websites and varying skill require- ments. Another emerging trend to adopt sampling or search algorithms for dynamic exploration of web navigation actions, which reduces dependence pre-defined strategies but increases the cost of LLM inferences (Koh et al., 2024b; Zhang et al., 2024; Pan et al., 2024).
In this work, we aim to enhance LLM-based web agent's proficiency by optimizing the text- based task understanding and reasoning of existing LLMs, rather than refining the agent strategies. Automating web tasks is challenging, as the agent needs to i) accurately extract information from web pages with varying formats and encoded scripts, and ti) issue appropriate embodied actions, selecting from those defined merely web (e.g., scrolling, clicking, or hovering over buttons). These web observation and action spaces are less common in both, the pre- and post-training data of LLMs, preventing the LLMs from fully realizing their potential in accomplishing general- purpose web tasks. Therefore, we study how to properly tune the observation and actions for LLM- web agents, align them with the functioning LLMs capacities learned during training.
As shown in Figure 1, our method comprises of three components: i) We reduce non-essential ac- tions to minimize the agent's embodiment and trivial interaction needs; ti) We refine the observation by eliminating redundant and irrelevant web elements, and restructuring web content blocks for more suceinet yet as informative representations; iii) We introduce two planning actions (branch and prune), which enables the agent to self-organize navigation workflow with a planning tree, and use the same structure to filter the previous traces for history replay. It's noteworthy that those plan- ning commands are in the same position as navigation prompts for the agent. We implement these components by generic rules that applies to all types of waw.pp-lnnuggg---oo without leveraging task-relater information the test benchmark.
web pages,
By combining the three techniques mentioned above, our proposed agent AGENTOCCAM per- forms substantially better web tasks across websites in the WebArena environments (Zhou al., 2023b). AGENTOCCAM outperforms the previous state-of-the-art approach by 9.8 absolute points (+29.4%) and surpasses concurrent work by 5.9 absolute points (+15.8%). Notably, unlike most prior work, we do not use any in-context examples, additional online search or sampling, nor spe- cialized prompting templates or agent roles play well. In contrast, AGENTOCCAM delivers such strong performance with an unexpectedly simple approach: letting the LLM issue actions within the processed and augmented observation and action spaces. Compared with similar plain web agent without these proposed observation and action space changes, AGENTOCCAM increases the suc- cess rate by 26.6 absolute points (+161%) 6).. Moreover, we prove AGENTOCCAM's web environment generalizability in the real-world web environment. In the Vobgee definite-answer subset (He et al., 2024), which consists of real-world web tasks with deterministic answers, AGENTOCCAM exceeds the previous best agent this benchmark by 2.4 points (+4.6%).
In summary, the primary contribution of this work are as follows. First, we develop a new of the-art agent, AGENTOCCAM, for web tasks. On the WebArena benchmark consisting of 812 tasks across five diverse websites (c.g.. shopping, searching on a forum). AGENTOCCAM outperforms previous and concurrent work significantly. Second, shed light on the strong zero-shot perfor-
2

---

Published as a conference paper at ICLR 2025
Table 1: Comparison of essential components for different LLM-based web agents.
Essential Components
Task-specific Strategies
In-context Examples
Additional Module
Offline Data*
Online
Search
AutoGuide (Fu et al., 2024)
NO
YES
YES
YES
NO
SteP (Sodhi et al., 2024)
YES
YES
YES
NO
NO
AutoRefine (Pan et al., 2024)
YES YES
YES
YES
LM-Tree Search (Koh et al., 2024b)
YES
YES
YES
AWM (Wang et al., 2024)
NO
YES
YES
YES²
NO
WebPilot (Zhang et al., 2024)
NO
YES
YES
NO
YES
Agent-E (A buelsaad et al., 2024)
NO
YES
YES
NO
NO
AGENTOCCAM
NO
NO NO NO
NO
mance of LLMs on web tasks with our simple agentic workflow, in sharp contrast to many more complex compound agent policies. Last, our work on aligning the observation and action spaces is orthogonal to agentic strategies and can be combined with future advances in that aspect.
2 RELATED WORK
LLM-based Web Agent Advances large language and multi-modal foundation models have significantly boosted the development of autonomous agents to solve web tasks. Techniques translat- ing LLMs to powerful web agents, inspiring many techniques that design inference time agent strategies. Many prior ap- proaches improve the agent system by designing auxiliary modules with specialized LLMS or roles, aiming break down complex tasks (Sun et al., 2024; Prasad et al., 2024; Abuelsaad et al., 2024). Other work leverages LLMS to extract common patterns from examples or past experience (Zheng al., 2023; Fu et al., 2024; Wang et al., 2024). However, this line of work often relies on pre-defined control hierarchy, prompt templates or examples to act accurately in the test environments. For ex- ample, SteP (Sodhi al., 2024) utilizes a stack-based approach for dynamic multi-level control in the web tasks but relies on task-specific atomic policies with environment--elated information hard- coded in prompt template. Another line of work focuses on improving web agents' performance by leveraging more online examples from the environments. Many of them (Zhou et al., 2023a; Zhang et al., 2024; Putta et al., 2024) adapt Monte Carlo Tree Search (MCTS) methods, expanding inter- mediate states (tree nodes) in one task repeatedly by multiple trials over that task. Among them, WebPilot (Zhang et al., 2024) also adds a global optimization layer for high-level planning. Koh guide search and to back-trace
berision-maaing agents (Yao et al., 2022b; Shinn et al., 2024) have vanced
et al. (2024b) use trained value function tree. Auto Eval and Refine (Pan et al., 2024) trains a separate evaluator, and improves the task execu- tion using reflective thinking (Shinn et al., 2024) on past trials in the same task. However, sampling or resetting multiple times in the same task, not only increases the inference cost significantly, but also limits its applicability when failed task is not revocable. As a comparison, we highlight the simplicity of our method and its difference with related agent approaches in Table 1.
the task execution
Fine-tuned or Trained Models for Web Tasks Fine-tuning language or multimodal models for web tasks is another effective approach to enhance decision-making capabilities on the web tasks (Yin et al., 2024; Hong et al., 2024; Lai et al., 2024; Putta et al., 2024). While fine-tuning promises more adaptivity and broader optimization space, the size of task-specific fine-tuned models are typically not comparable with the most powerful closed-source models. There is also some early research that trains models to follow natural language command the web before L.MMs emerged, using semantic parsing (Artzi & Zettlemoyer, 2013), reinforcement learning (Branavan et al., 2009) and imitation learning (Liu et al., 2018; Humphreys et al., 2022). However, those fine-tuned agents, limited by the base model's 's capacities or training data volume, often fail to match those constructed with L.LMS regarding performance or/and generalizability, and is beyond the scope of this work.
Simulated Web Agent Environments Web agent development has been supported by increas- ingly complex web simulators for training and evaluation. These range from basic platforms like MiniWoB (Shi al., 2017) and its extension MiniWoB++ (Liu al., 2018), to more sophisticated
The offline data refers to labeled dataset to instill human knowledge into models. 'WW supports two scenarios: in offline scenarios it directly leverage offline dataset, and in online scenarios it relies domain-specific evaluator from Pan et al. (2024) which requires offline data train.
3

---

Published as a conference paper at ICLR 2025
environments such WebShop (Yao et al., 2022a), WebArena (Zhou et al., 2023b), and Visual- WebArena (Koh et al., 2024a). These simulators progressively incorporate weal-world complexities, from simple form-filling to tasks across multiple full-featured websites. In this work, we focus only on the text modality, and use WebArena to evaluate our method's task success and generalizability as it contains different types of websites and task-intents in single suite. To further assess AGEN- TOCCAM's ' generalizabilty. we extend experiments to the real-world web environments, evluated with the tasks and golden answers proposed in WebVoyager (He et al., 2024).
3 PROBLEM FORMULATION
We formalize the web interaction process by Partially Observable Markov Decision Process (POMDP, Littman (2009); Spaan (2012)): (O, S, A, P. R, Po, 7)- In POMDPs, an observation O consists of information that the agent receives from the web environment, e.g. HTMLs, as well as any instructions and prompts. In this work, we only consider the text modality. A state E S denotes the whole underlying (unobserved) state of the agent and the environment such that the state transition is Markovian. An action E AA is either a command recognized by the web en- vironment, or any other unrecognized token sequence that will lead a stay in the current state. P denotes a deterministic state transition function that records the change in the webpage state given the current state and agent action. R is the reward function that decides the success or failure of the agent's sequence of actions. Po denotes the initial state distribution which is uniform over tasks tested. The discounting factor y is typically set to when the reward is only assigned at the end of an agent-web interaction episode.
To solve POMDP, a common goal is to find a decision policy T(ag)h) (e) maximizing the expected cumulative reward, where h4 denotes the observation history {ou, 01,..., 02). In LLM-based web agent design, that is translated to designing a policy x T(a h2) with the help of one or more base LLM policies ILLM and a of algorithmic modules. In this work, we work on special class of policies that can be expressed as: (g((a))))) = TLLM (a where / and g are rule-based functions that process the observation (including action instructions) and actions for the LLM policy. We name the observation and action space alignment problem. Notice that under such a problem setting, all of our changes apply only to the observations and the actions. We emphasize not all agent strategies in previous approaches can be represented in this way. For example, search-based algorithms require a control program on the top to select actions and trigger back-tracing; methods with evaluators, reflective thinking memory modules also necessitate managing center to alternate between the main LLM and these helper segments or other role-playing LLMs. In contrast, we aim to answer the following question in our work: Can we build strong web agent with the base LLM policy TLL by optimizing only the observation and action mapping f and g?
METHOD
Rather than introducing any new modules or hicrarchical structures on top of the base LLM, our method focuses on simple web agent workflow that inputs the web observations to a general- purpose LLM-API and uses the LLM outputs as actions directly. In this section, we describe the process of aligning web tasks, which necessitates embodiment knowledge, with the predominantly static and text-centric nature of LLM training. Section 4.1 discusses our strategies (summarized in Figure for refining the action space to be more compact and reducing the need for the agent's embodiment capabilities. Section 4.2 outlines our methods (summarized in Figure 4) for condensing web content descriptions to be both brief and informative, and identifying key web elements and relevant steps for retention to organize the agent's 's memory in a pertinent manner.
4.1 ACTION SPACE ALIGNMENT
web agent's action space defines the valid commands can use to interact with the web envi- ronment. The WebArena simulator supports translating three categories of actions into mouse and keyboard operations: basic actions (c.g., click, type), tab operations (c.g., tab_ for man- aging active tabs), and page operations (e.g., backack for navigation). These actions are detailed in Appendix A, along with a comparison of our changes to the action space.
Based on observation of common failure modes in web agents, there exist two key challenges to be addressed by editing the action space: i) removing irrelevant actions that LLMs struggle to understand and frequently misuse, and äi) improving the memorization and planning ability when
4

---

Published as a conference paper at ICLR 2025
Action Space Alignment
3. Plan via Generation
Low-Embodiment Actions
Cbjeclive
Obiective
Original Actions
diick
acTol
bpp
go back
Cbjecliva
Pan
Plan
Basic Actions
noop
click
note
homee
Plan
Plan 2
Plan
hover
type
press
scroll
Planning Tree
branch
2. Disable Scrolling
Tab Operations
Isb focus
Avoid arberay sooliig.
Package aomil adiion Nigheniteell opecation..
nitw Tab
Tab closa
Core Navigation Actions
Essential Actions
ciick
type
go_ gocback
go_ Johoeee
For Navigation
Page Operations
go_
bpe
back
nota
stop
For Workllow Management
go.. Forwardd
golo
note
stop
go homee
Planning Actions
PONy In mulsite tosks
branch
Improved Decision-Making
1. Remove Non-essente Actions
In aligning the action space with LLM pre-training, we only retain high-utility actions and lessen the demand for advanced embodiment skills (steps and 2). Additionally, we incorporate planning steps, allowing the agent autonomously manage task breakdown and execution (step 3).
Figure 2:
the task execution requires navigating multiple potential paths to succeed. We propose that the first can be corrected simply by removing and combining actions. The second issue was often addressed in the previous work using handcrafted rules or strategies, making these approaches hard to generalize. In this work, we tackle the second problem by introducing actions that allow the LLM to autonomously generate plans and manage the task workflow. These proposed solutions are explained in detail below and illustrated in Figure 2. The list of all actions in original and reduced action space is shown in Table 3, together with the frequency they are taken in different agents.
Simplifying the Action Space. First, we eliminate actions that can be replicated using similar ac- tions or replace multiple actions with one action with the same expressiveness (illustrated in Figure step 1). Specifically, we remove the noop action, signifying "no operation", as it is shown to be a distraction to the agent in most cases. Similarly, tab operations, which manage the focusing, open- ing, or closing of tabs are removed because they are only needed in limited cases of multi-site tasks requiring two tabs. Furthermore, we limit page navigation actions like go_forwa rd and goto, as their utility is greatly constrained by the agent's 's poor memory of the relationship between page's URL and its content. By eliminating these less effective actions, our goal is to minimize distrac- tions and boost the agent's concentration on more meaningful operations. In addition, we introduce the note action, allowing the agent to record key observations for subsequent conclusions, and the s stop action, enabling the agent to autonomously conclude the trajectory with answers. We also add a go_home command for multi-site tasks, enabling the agent to navigate directly to the homepage where all available sites are listed.
Second, we eliminate actions that heavily require embodiment knowledge and simplify low-level actions into more abstract operations as shown in Figure 2 step 2. In particular, we reduce commands that LLMdb agents struggle with unless provided with detailed context-specific guidance, like hover or press (the latter is for pressing key combinations, often shortcuts). To properly use these actions requires LLMs to have embodied thinking of the current scenario, especially regarding the mouse position or keyboard operations, which it has not acquired during their training. Additionally, we remove the scrol] 11 action, opting instead to load the full page content as the web state. This change is in response our observation that agents tend to engage in aimless and repetitive scrolling when an essential link is not visible at the top of the page, wasting steps without making progress. Furthermore, we streamline the agent's interaction with drop-down menus; instead of selecting the menu and then an option, single click command with the ID of the desired option now suffices.
Planning via Generation. Web tasks often require a solution that requires navigating multiple paths, e.g. extracting information from one page and submitting it to another page, like the task of creating a refund request on the Contact Us page for a broken product (task template 154), which requires parsing the order ID and refund amount from the order pages. We introduce two actions, branch and prune, to generate plans in a tree structure and save them for future observations. As Figure 2 step shows, the LLM generated plans start with root node being the objective of the task. The branch action will generate new subplans under the current node, decomposing high-level objectives into smaller, more manageable subgoals. Conversely, the prune action allows the agent
5

---

Published as a conference paper at ICLR 2025
General Instruction
Task Description
Egg. You anw sesccrred web nevigator: You need iseua en action for this step...
Agent
Output Spec fncsionn
Reasen: Frovide callonals proposing
commands, Action:
Prompt
Action Speciiicaions
E.g. dlick [id): To click on on doonent in numerica on the web page.
Online Task
Slep
Objective
Previous Interaction
Current Observation
Page Lengith
History Length
Figure 3: The components of our web navigation agent's prompt. It includes general instruc- tion outlining the task, the desired output and available actions, well as online task information providing the current goal, the agent's past interactions, and the latest observations. Notably, the sections on previous interactions and current observation use the most tokens, and can be attributed to two main factors: the length of the pages and the extent of history span.
Observation Space Alignment
Original Environment Description
Original Web Content Block
1. Simplify Web Page Elements
Hoaf/teb<rma [I] Wretess Headphoned (Poqused: True]
man
[1912] Account
(761) My
caplic
Refined Web Environment
(15110)
Description
Remove rodundant text.
Staticleet items
See ChachTee CCrttr
RouWWibArca 'Wollss Headphones!
Iink
Out
Ink Account"
StaticTooo (764) 'Sign Out
Ink [1515] My Mish in Heme!
Staticllst [789] WWetoom, Emnna Lopec?
Saa "Oeer Touu
Ink [1314] Sign Out
jeqcined Faiii Stacceen "rake"
text Welcome, Emma Lopoz!
colrriheade (rouured: Tated)
Natton
Refined Web Content Block
2. Convert HTML accessibiilt tree to Markdown.
gricial chancoser yoqulwe Friee)
gricud Sea Vichiied
Seectee voc0oittin
now Onder Date (Oedrr lotdl States (Aciion
Stal CTale 10-000
now (000000192 621.04 8,369 Ss| Pranding View Orde link (463s0) 'Vicx
isa siee "hayee
iea deead "Now Orier
Selectively Replay Web Elements in One Page
Selectively Replay Past Pages
Planning Tree
Step
Invisible step
Tree-Structured Web Elements
Pivotal node lsg. lod Vie Orerr)
Plan
Stop
Visible stop
Ancestor node eeg. ink Ordrer))
Sumplan
Step
Step
Step
Sibling node led 174242)
Descendent node lik view Ordert
Subplan 2
Step
Current stop
Subsubpian
Step 5
Stap
Figure 4: To align the web task's observation space with the format that the base model perceives most effectively, we condense a single-page page length by removing unnecessary texts that repetitively describe the web page's 's functionality and layout (step 1), and by identifying page elements rele- vant to the task for the agent to remember (step 2). Additionally, we optimize the agent workflow memory through planning tree, viewing each new plan as a separate goal and excluding past steps' information dedicated to previous plans enhance memory conciseness (step 3).
give up the current sub-plan, often after repetitive failed attempts, and seek alternatives. Together with the branch and the prune actions, the LLM can edit the planning tree autonomously. Note that these two planni ng actions are of no difference from the native navi gat on actions in the web environment (e.g. click, type) and the LLM is free to choose when to take these actions to update the plan. The generated plan provides a context for future action generation and enhances the consistency of actions in one trajectory. This approach leverages the intrinsic planning ability of LLM itself. We argue that this will not compromise the agent's generalization capability as this design relies minimally on task saec-fpe prior knowledge.
4.2 OBSERVATION SPACE ALIGNMENT
The observation space of web agents consists of task objectives, instructions, previous interaction, and the current web text descriptions or screenshots (see Figure and Appendix for our agent). Among them, previous interactions and current web content consume the most number of tokens, which scales with the length of a single page and the length of history. This often results in a long context window, which not only increases the LLM inference cost but also poses challenges for LLM to extract related information accurately. Therefore, our main goal in refining the observation address these two aspects. The alignment of observations is outlined in Figure 4.

---

Published as a conference paper at ICLR 2025
Simplifying Web Page Observations. The content on web pages is represented in HTML cessibility tree format in most text-only web agents. These formats are designed towards front-end loading and rendering, containing numerous formatting tokens making them lengthy and repeti- tive, as illustrated in Figure Step 1. Our goal is to optimize the representation to make it more readable to LLMs on one single page. Specifically, we merge function-deserripivy web elements (e.g., StaticText [761] 'My Account" ) with interactive elements that share the same la- bel (e.g., link [1312] 'My Account') ). We then convert table and list blocks Markdown, eliminating repetitive structural tokens (e.g., co 1 umnheader, gridcell). Consequently, we achieve a more concise representation while keeping the same information.
Replaying Observation History Selectively. Taking observation history as input is important for decision-making agents to act consistently for tasks requiring long horizons, given that the obser- vation state only contains partial information about the environment's 's state. For web tasks, it is also important to include both observation and action history as some key information may not be displayed on the current page. However, the observation history will also significantly scale up the context length and increase reasoning difficulty as well as inference cost. We address this issue by only selecting the most important and related information two rules based on the "pivotal" nodes (defined later) and the planning tree. We provide detailed examples of how these two techniques are implemented in Appendix B.
the previous web pages, according to
First, we observe that only a small amount of content on a web page is pertinent to a specific task among several steps and is worth replaying in future steps. For example, in tasks requiring the agent to find all reviews within three months, it is unnecessary to keep other reviews or some unrelated links like Contact small amount of content by leveraging the tree structure of web data (e.g. accessibility tree). To do this, we first instruct the agent to pinpoint the crucial web elements denoted as "pivotal" nodes, at the same time when the agent generates an action. The agent is then programmed to include only the pivotal nodes' ancestor nodes (indicating their global hierarchy and position), sibling nodes (providing immediate context), and descendant nodes (offering detailed characteristics) in the future observations as illustrated in Figure 4 Step 2. This effectively narrows down the volume of data and level of noise passed to the future context of LLM inference.
on the page. Thus we employ a simple rule to identify this
Second, we observe that not all previous steps' observation needs to be noted during the inference of future steps. Thus can leverage the planning tree generated by the agent itself to keep the agent's 's focus sharp. Specifically, when the agent initiates a branch action to develop a new plan, treat this new plan separate goal. Steps taken for earlier plans and their observations will be dismissed in the current plan's 's observation window, as depicted in Figure 4 step 3. This allows the agent focus only information dedicated to the current plan for sub-task.
5 EXPERIMENTAL RESULTS AND ANALYSIS
Here, we detail experiments WebArena (Zhou et al., 2023b), web simulator benchmark. Further experiments with WebVoyage (He et al., 2024), a web benchmark based on real-world websites, are included in Appendix C. We show AGENTOCCAM' 's base model generalizability in Appendix D.
Environment. We utilize WebArena, an interactive web simulator, as our benchmark. WebArena consists of fully functional websites from four common domains: e-commerce platforms (OneStop- Shop), social forums for idea and opinion exchange (Reddit), collaborative software development (GitLab), and content management for creation and management of online data (online store man- agement). The platform additionally includes utility tools: a map, a calculator, a scratchpad, and Wikipedia to enable human-like task-solving. The benchmark consists of 812 tasks generated from 241 templates. A template here is a parametric form of a task intent, allowing for multiple instan- tiations with different keywords. Each task is accompanied by evaluator that programmatically checks the correctness of the final information with respect to the desired ground truth information". We use GPT-4 --uboo 2024 00--0- (Achiam et al., 2023) build our AGENTOCCAM.
Baselines.
We compare AGENTOCCAM with the following prior and concurrent work: 1) We- bArena agent: the Thouuht (CoT) prompted agent included in the WebArena benchmark
'We identified and corrected errors in the original evaluators, with details discussed in Appendix E, Our approach outperforms the baseline methods with both the original and the corrected evaluators.
7

---

Published as a conference paper at ICLR 2025
Table Comparison of the success rate (SR) of AGENTOCCAM with baseline agents on WebArena.
Agent
Model
(5E)
Shopping
Shopping Admin
GitLab
Mup
Reddit
Multisite
(8812)
(#187)
(#182)
(#180)
(#109)
(#106)
(#48)
WebArena-replication
GPT-4-Turbo
16.5
16.6
15.9
10.0
22,9
21.7
16.7
SteP-replicution
GPT-4-Turbo
33.3
33.2
32.4
26.7
35.8
52.8
12.5
AWM
GPT-4
35.5
WebPilor
GPT-4o
37.2
AGENTÖCCAM
GPT-4-Turbo
43.1
40.6
45,6
37.8
46.8
61.3
14.6
Vanilla
Above + X Scrolling
Above + History
J Actions
Above Obs Opt.
AgentOccam (Above Planning)
Success Rate
All
Shopping Shopping admin
Gitlab
Map
Reddit
Multisite
Figure 5: Ablation study of AGENTOCCAM'S action and observation space alignment, with details
in Table 17. We incrementally add refinement components and assess marginal performance gains.
(Zhou et al., 2023b). 2) SteP (Sodhi al., 2024): a stack-based approach on top of 14 human- atomic strategies tailored to solving WebArena. 3) WebPilot (Zhang et al., 2024): a multi-agent, multi-level MCTS based agent that reports state-of-the-art overall performance on WebArena. 4) Agent Workflow Memory (AWM) (Wang et al., 2024): method automatically summarizing work- flow from past experience. SteP has made its code and interaction trajectories public. Hence, we are able to fully replicate the agents from WebArena and SteP with EPT-4-turbe in the identical web environments as our methods, for a fair comparison." WebPilot and AWM, being concurren works with this paper, have yet provided source code resulting trajectories, limiting our anal- ysis of these works to just reporting the aggregated performance numbers included in their technical reports. Our analysis focuses SteP it was the most performant method prior to this work.
Question 1: How well does AGENTOCCAM perform? As seen from the results Table 2, our agent AGENTOCCAM, which optimizes the action and observation space, sets on the WebArena benchmark. increases the overall success rate from 37.2% to 43.1 %, a 15.8% relative improvement over best results among previous and concurrent work. We observe that AGENTOCCAM not only accomplishes tasks in the template that is previously unsolvable, like up- dating personal information for templates with mixed results previously, such as setting a homepage URL on a GitLab profile (task template 331). This is further illustrated in Figure 6 in the appendix.
SOTA
SheStopShop (task template 165), but it also raises the success rate
How much does each observation and action space change contribute to AGEN- TOCCAM? We evaluate the contribution of each component in AGENTŌCCAM described in Sec- tion to its overall success by incrementally integrating them into the vanilla agent (WebArena- Replication) and assessing the marginal performance gain shown in Figure 5. The details of each incremental experiment are as follows:
Question 2:
Removal of non-essential actions (1 Actions): Narrowing the action space reduce the level of distraction for LLM policies and significantly improves performance across all tested web- sites as shown Figure 5. By removing rarely used actions like sab_focus.go.forree hover and press, the agent spends fewer steps wandering around and explores more efficiently using actions such as click and type. Table 3 shows reduces hundreds of hover and goto actions while significantly increasing the number of click ck and type.
ti) Disabling scrolling (Above + X Scrolling): We observe that LLM policies tend to use scroll 1 up and down often when they do not know what to do (since these actions are revertible). Consequently, it significantly delays the task execution and causes looping in certain tasks. As
'In our experiments, we note that all agents occasionally fail due to emors from the WebArena simulator, such as posting rate limits in Reddit or login expiration. In such cases, restart the experiments.
8

---

Published as a conference paper at ICLR 2025
Table 3: Action statistics for the ablation study of AGENTOCCAM's components. Each number in the table represents the frequency of an action across all the tasks within the experiment setting. Actions noop, go.forward, tab and tab_close are not included since they are not used even once in vanilla agent and removed in our method.
Exp.
hover
type
press
scroil
new go.
goto
note
stop
go. go.home branch
prune
Vanilla
2328
126
1024
132
20
71
511
_2
Actions
7119
2531
370
52
194
512
36
Above 1. X Screlling
7033
2390
100
219
536
42
Above obs Opt.
6890
2040
56
201
571
23
Above + siatory
4625
1286
94
112
801
54
AGENTÖCCAM
4720
1159
339
197
769
42
34
47
Table 4: Average observation tokens per step across WebArena sites. We use the GPT2 tokenizer from HUGGINGFACE (Radford al., 2019).
Exp.
All
Shopping
Shopping Admin
GitLab
Map
Reddit
Multisite
Vanilla
2210.2
2272.1
2460.2
2199.1
1885.2
2132.4
1751.0
Act ians
1652.0
1644.7
2133.1
1981.3
912.0
1081.2
1296.8
Above X Scrolling
3376.2
3148.0
5403.7
3364.9
1378.1
2603.6
1975.5
Above 1 Obe Opt.
2891.1
1722.5
4791.7
2560.8
1476.4
3332.3
1619.4
Above History
3051.3
1802.6
5140.2
3153.3
862.1
3156.1
20300.3
AGENTÖCCAM
2930.9
1634.2
4920.7
3126.8
1056.0
3697.8
1282.5
result, disabling the scrolling action and passing the entire page to the agent proves advantageous, especially for GitLab and Reddit tasks. However, this strategy increases the number of observation tokens, which will be addressed by subsequent refinements.
tii) Simplifying web page elements (Above Obs Opt )): We remove redundant text and web format as show in Figure 4 Step 1. This results in fewer tokens in the context window, as outlined Table 4. helps the agent focus on web elements crucial to task success across all websites and boosts the performance on all task types, except on GitLab, where this sometimes leads the agent to overlook simpler solutions (task id 394).
iv) Selective replay of web elements in one page (Above + History): In this experiment, follow step 2 shown in Figure 4 to add a subset of elements from previous web pages as history. We observe that it allows the agent to avoid repetitive actions in tasks, significantly decreasing the steps needed for task completion as demonstrated in Table 5. However, this addition slightly hurts performance in tasks with dense single-page content those requiring navigation across multiple pages, as shopping and Reddit tasks success rate drops by 3.2 and 6.0 points, respectively.
v) Planning via generation and selective replay of past pages (AGENTOCCAM; Above Planning): We introduce actions branch and prune to allow the agent to autonomously gen- erate plans and exclude historical steps not in the current sub-plan from the prompt context. This results in performance gains in tasks across nearly all websites, alongside reduction in the required observation tokens. The actions branch and prune are both primarily used in correcting failed strategy and trying an alternative path. For example, in the task of identifying the nearest national park Boston (task id 265), the agent employs a branch action to adopt an alternative search strategy after a failed search attempt. In Gitl ab task (task id 563), after multiple failed attempts using the Create project button, the agent opts for a prune action to explore other methods.
Question 3: Could the power of AGENTOCCAM be combined with other agentic strategies? natural question ask next is if we can combine these changes with other common agent strate- gies or prior work, since the changes in observation and action space are orthogonal and comple- mentary to them. We showcase two example studies to answer this question: one with the SteP method (Sodhi et al., 2024) and another action selection method with LLM-as-a-juuge.
The judge method is motivated by our observation of the high variation the agent's behavior. In some key steps, the agent has certain probability of generating the correct action but often fails to do so, making it hard for the agent to recover from later pages. For instance, when tasked with identifying the most suitable subreddit for posting (task template 6100), the AGENTOCCAM agent
We remove stop in the statistics for the vanilla WehArena agent this action is excluded in their officially defined action space. However, their agent allowed by code to generate op to end the trajectory.

---

Published as a conference paper at ICLR 2025
Table 5: Average number of steps per task across all WebArena sites.
Exp.
All
Shopping
Shopping Admin
GitLab
Map
Reddit
Multisite
Vanilla
6.2
6.2
6.6
5.9
5.7
1.4
4.4
Actions
13.3
10.6
14.3
14.8
11.9
15.2
13.7
Above x Scrolling
12.7
9.0
14.0
14.8
12.7
13.0
14.0
Above abs Opt.
12.0
13.2
15.4
10.2
12.1
13.2
Above + Blatory
8.6
5.6
9.6
10.3
8.3
7.6
12.9
AGENTOCCAM
Table 6: Success rate (SR) of AGENTOCCAM combined with agent strategies on WebArena.
Agent
Model
(%)
Shopping
Shopping Admin
GitLab
Map
Reddit
Multisite
(#812)
(#187)
(#182)
(#180)
(#109)
(#106)
(#48)
AGENIOCCAM
GPT-4-Turbo
43.1
40.6
45.6
37.8
46.8
61.3
14.6
AGENTOCCAM SteP
GPT:4-Turbo
41.1
46.5
36.3
36.7
47.7
50.9
18.8
AGENTOCCAM Judge
GPT-4-Turbo
45.7
43,3
46.2
38.9
52.3
67.0
16.7
tends hastily choose less relevant subreddits and gets stuck there. To address this, direct the AGENTOcCAM to generate all possible suitable actions instead of one action at each step. These action candidates are then evaluated by another LLM (GPT-4--urte the role of a judge and select the best action. The prompts for the judge are included in Appendix G.
as well) prompted play
Table shows that AGENTOCCAM SteP agent, enhanced with task strategies, outperforms the standalone SteP method but doesn't 't match AGENTOCCAM 's base performance. Additionally, com- bining AGENTÖCCAM with a judge role through an action prediction and selection pipeline rectifies some of the base agent's behavioral misconduct.
By analyzing the trajectories of each method, we observe that task-specific strategies like those introduced in SteP can help when the strategy fits the task requirements. For example, in the task of "Draft an email to the shop owner via their contact us functi ion {reason}" (task template 163), the AGENTOCCAM
for a coupon agents excel by prompting the agent explicitly not to click the submit button after drafting, where AGENTOccAM fails to follow. However, for tasks outside the designed strategies, these hints can mislead the agent, leading to a 2-point drop the overall success rate of AGENTOCCAM SteP compared AGENTOCCAM only. An example is task 639, where the agent, guided by SteP's instruction "Under forums, you will see only a subset of subreddits. get the full list of subreddits, you need Alphabetical option. ", repetitively navigates away from the appropriate subreddit, and generates reasons for its action selection that "Clicking on the "Alphabetical" link will help us access a more comprehensive Reddit list.", demonstrating how hard-coded strategies can distract the agent and hurt generalizability.
SteP and SteP
explicitly
navigate
the
The AGENTOCCAM Judge agent, combining the AGENTOCCAM's generated action list with the second opinion from an L.LM judge increases its overall success rate by 2.6%. by completing tasks where it may well fail due to intermediate decision flaws. For example, in choosing the right sub- reddit for post (task template 6100), the base AGENTOCCAM might hastily pick from an initial list, whereas the AGENTOCCAM Judge agent conducts thorough search using post keywords or explores the entire forum list before drafting the post. This approach minimizes errors due to rushed decisions, increasing the likelihood of successfully completing task series.
CONCLUSION
In this paper, we proposed simple but effective LLM-based web agent AGENTOCCAM that refines its action and observation spaces be more comprehensible for L.LMS primarily trained on static text. Unlike other methods, AGENTOCCAM stands out for its remarkably simple policy workflow, requiring no extra modules, additional LLM calls, or in-context examples. This simplicity does not compromise its performance; AGENTOCCAM surpasses previous and contemporary approaches on WebArena by 9.8 (SteP) and 5.9 (WebPilot) absolute points, respectively. Our results emphasize the importance of maintaining simple agent architecture for better generalizability, echoing Occam's razor principle. In summary, AGENTOCCAM aims to lay solid foundation and offer valuable insights for future web agent research and development.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Tamer Abuelsaad, Deepak Akkil, Prasenjit Dey, Ashish Jagmohan, Aditya Vempaty, and Ravi Kokku. Agent-e: From autonomous web navigation to foundational design principles in agen- tic systems, 2024. URL /tttpp://axxi..oo org/ abs 2000./3332..222.
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, 1ige Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. Xrviv preprint arXiv:2303.08774 2023.
Yoav Artzi and Luke Zettlemoyer. Weakly supervised learning of semantic parsers for mapping instructions to actions. Transactions of the association for computational linguistics, 1:49-62, 2013.
Satchuthananthavale RK Branavan, Harr Chen, Luke Zettlemoyer, and Regina Barzilay. Reinforce- ment learning for mapping instructions to actions. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pp. 82-90, 2009.
Yao Fu, Dong-Ko Ki Kim, Jaekyeom Kim, Sungryull Sohn, Lajanugen Logeswaran, Kyunghoon Bae, and Honglak Lee. Autoguide: Automated generation and selection of state-aware guidelines for large language model agents, 2024. URL http:: //arxii///axvv.../// www.iivvvrrg/aaae org/abs 2403.08977
Chen Gao, Xiaochong Lan, Nian Li, Yuan Yuan, Jingtao Ding, Zhilun Zhou, Fengli Xu, and Yong Li. Large language models empowered agent-based modeling and simulation: survey and perspectives. Humanities and Social Sciences Communications, 11(1):1-24, 2024.
Google. Gemini-1.5, 2024.
URL https://blog httggggggggl/eeoooloooooo
Snoogle-geminie mimriii-ai/A /awaaii//snaaar-aaaaa note.
Hongliang He, Wenlin Yao, Kaixin Ma, Wenhao Yu, Yong Dai, Hongming Zhang, Zhenzhong Lan, and Dong Yu. Webvoyager: Building an end-to-end web agent with large multimodal models, 2024. URL httttps:/arxxv.orrg
2ttpb////411133 13919.
Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxiao Dong, Ming Ding, et al. Cogagent: visual language model for gui agents. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp-
14281-14290, 2024.
Peter C Humphreys, David Raposo, Tobias Pohlen, Gregory Thornton, Rachita Chhaparia, Alistair Muldal, Josh Abramson, Petko Georgiev, Adam Santoro, and Timothy Lillicrap. data-driven approach for learning to control computers. In International Conference on Machine Learning, pp. 9466-9482. PMLR, 2022.
Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Lim, Po-Yu Huang, Graham Neubig, Shuyan Zhou, Russ Salakhutdinov, and Daniel Fried. VisualWebArena: Evaluating multimodal agents on realistic visual web tasks. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.
Bangkok, Thailand, August 2024a. As- sociation for Computational Linguistics. doi: 10. 18655/v1/2024.acc s/aclantholos org/ 2024.........2.. acl acl 11l-1on..500
URL https:
Jing Yu Koh, Stephen McAleer, Daniel Fried, and Ruslan Salakhutdinov. Tree search for language model agents, 2024b. URL htttp:://arxi......//// wwwpa/wwwwr////oooo 2200b//244.000 01466.
Hanyu Lai, Xiao Liu, Iat Long long, Shuntian Yao, Yuxuan Chen, Pengbo Shen, Hao Yu, Hanchen Zhang, Xiaohan Zhang, Yuxiao Dong, al. Autowebglm: A large language model-based web navigating agent. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 5295-5306, 2024.
Michael L Littman. A tutorial on partially observable markov decision processes. Journal of Mathematical Psychology, 53(3):1119-13 2009.

---

Published as a conference paper at ICLR 2025
Evan Zheran Liu, Kelvin Guu, Panupong Pasupat, Tianlin Shi, and Percy Liang. Reinforcement learning on web interfaces using workflow 2018.
wrefflow-gidedd guided exploration. arXiv preprint wtXivv11822.0820
Jiayi Pan, Yichi Zhang, Nicholas Tomlin, Yifei Zhou, Sergey Levine, and Alane Suhr. Autonomous evaluation and refinement of digital agents. In First Conference on Language Modeling, 2024.
Archiki Prasad, Alexander Koller, Mareike Hartmann, Peter Clark, Ashish Sabharwal, Mohit Bansal, and Tushar Khot. ADaPT: As-needed decomposition and planning with language mod- els. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Findings of the Association for Computational Linguistics: NAACL 2024, pp. 4226-4252, Mexico City, Mexico, June 2024. Association for Computational Linguistics. doi: 10.18653/v1/200 1/2.244..ndinngs...... www.diings-nnaa..666.... naac1.264. URL https: aaclllllll http///aaanttooooy.oooooo org/ 2024. fi
244.
Pranav Putta, Edmund Mills, Naman Garg, Sumeet Motwani, Chelsea Finn, Divyansh Garg, and Rafael Rafailov. Agent q: Advanced reasoning and learning for autonomous ai agents. arXiv preprint arXiv:2408.07/ 00..007119 2024.
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. OpenAI, 2019. URL https: / h/thghghgggggn....e./ co/ copoopeni--oe communi y /pt2..
Tianlin Shi, Andrej Karpathy, Linxi Fan, Jonathan Hernandez, and Percy Liang. World of bits: An open-domain platform for web-based agents. In International Conference on Machine Learning, pp. 3135-3144 PMLR, 2017.
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36, 2024.
Paloma Sodhi, SRK Branavan, Yoav Artzi, and Ryan McDonald. Step: Stacked Ilm policies for web actions. In First Conference on Language Modeling, 2024.
Matthijs TJ Spaan. Partially observable markov decision processes. In Reinforcement learning: fafte-of-the-atee pp. 387-414. Springer, 2012.
Haotian Sun, Yuchen Zhuang, Lingkai Kong. Bo Dai, and Chao Zhang, Adaplanner: Adaptive plan- ning from feedback with language models. Advances in Neural Information Processing Systems, 36, 2024.
Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, and Graham Neubig. Agent workflow memory, 2024. URL pettps://axxi..org/aatttte /ww.///.///0222222
Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et The rise and potential large language model based agents: A survey. arXiv preprint 2xXxv:2309.07864 2023.
Ke Yang, Jiateng Liu, John Wu, Chaoqi Yang, Yi R Fung, Sha Li, Zixuan Huang, Xu Cao, Xingyao Wang, Yiquan Wang, et al. If I1m is the wizard, then code is the wand: A survey on how code empowers large language models serve intelligent agents. arXiv preprint arXiv:2401.000 .00812, 2024.
Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. Webshop: Towards scalable real-world web interaction with grounded language agents. Advances in Neural Information Processing Systems, 35:20744-20757, 2022a.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.0362 2022b.
12

---

Published as a conference paper at ICLR 2025
Da Yin, Faeze Brahman, Abhilasha Ravichander, Khyathi Chandu, Kai-Wei Chang, Yejin Choi, and Bill Yuchen Lin. Agent lumos: Unified and modular training for open-source language agents. In Lun-We Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 12380-12403, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 10.18653/v1/ 1124.acl-long6700 URL ht hataaaaaannntoooooooooooeoo 2024 wwl.o22.aclooonoooo 670.
Yao Zhang, Zijian Ma, Yunpu Ma, Zhen Han, Yu Wu, and Volker Tresp. Webpilot: A versatile and autonomous multi-agent system for web task execution with strategic exploration, 2024. URL.
https://arxiv. httgg/bs/2408 225..8588888
Longtao Zheng, Rundong Wang, Xinrun Wang, and Bo An. Synapse: Trajectory-as-exempee prompting with memory for computer control. The Twelfth International Conference on Learning Representations, 2023.
Andy Zhou, Kai Yan, Michal Anlapentokh-Rothman Haohan Wang, and Yu-Xiong Wang. Lan- guage agent tree search unifies reasoning acting and planning in language models. arXiv preprint 1rXiv:2310.04466, 2023a.
Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng. Tianyue Ou, Yonatan Bisk, Daniel Fried, et al. Webarena: A realistic web environment for build- ing autonomous agents. arXiv preprint arXiv:2307. 13854, 2023b.
13

---

Published as a conference paper at ICLR 2025
A
COMPARISON OF THE VANILLA AND THE ALIGNED ACTION SPACE
Table 7: The action space in WebArena.
Category
Action Type
Description
noop
Do nothing
click (elem)
Click at an element
hover (elem)
Hover element
Basic Actions
type (elem, text)
Type an element
preas (key_comb)
Press key combination
scroll (dir)
Scroll up and down
tab focus (index)
Focus on the i-th tab
Tab Operations
new.tab
Open a new tab
tab close
Close current tab
go.back
Visit the last URL
Page Operations
go.forward
Undo go. back
goto(URL)
Go to URL
Table 8: The aligned action space of AGENTOCCAM.
Category
Action Type
Description
toop
De-nothing
click [id]
Click at an element
hovez
Hover en
Basic Actions
type [id] [content] ]
Type to an element
press
eerell
Sereil ueplll-u--nnee dewn
Feeus-o ihe th
Tab Operations
Open a new tab
tab-elese
Close enrreee tub
golback
Visit the last URL
formadd
Undo-go
Page Operations
gste
Ge-
go_home
Go to home page
note [content]
Take notes
Workflow Management
stop [answer]
Stop with answer
branch [id] [intent]
Generate a new plan
Planning Actions
prune [id] [reason] Restore to previous plan
We list the action space of WebArena and our aligned action space in Table and 8, respectively. In detail, we remove non-essential and uuddiment-underrsaddinngg.g scroll, and add more actions for internal workflow management or autonomous planning control.
actions like noop and
B PIVOTAL NODES AND THE PLANNING TREE CLARIFICATIONS
To prevent confusion, we separate the following explanations for the independent techniques of "selectively replaying web elements on page" and "selectively replaying entire pages."
Valid only in multisite tasks.
14

---

Published as a conference paper at ICLR 2025
B.1 SELECTIVELY REPLAYING WEB ELEMENTS ON A PAGE WITH PIVOTAL NODES (FIGURE STEP 2)
B.1.1 INTUITION AND BACKGROUND
We could view selectively replaying web elements on a page as a focused memory mechanism, where only the information that's relevant the task would be recorded and replayed the agent history traces. As all the web pages could be framed into a (DOM) tree structure, any web elements are nodes in this representation. We define pivotal nodes as the web elements, be it interactable or not, that are potentially useful for completing the task. Those pivotal nodes might present infor- mation (e.g., customer reviews navigate crucial states (e.g.. the search box and the search button in search task). We obtain these nodes at each step by prompting the agent to generate the page's 's highlights based on which they issue the action, or any web elements they will attend to if they fail at future steps and restore to the current state. After identifying the pivotal nodes with the agent's s efforts, our code supports automatically parsing the web page's DOM tree and retaining nodes that are associated with the pivotal nodes (i.e., pivotal nodes' ancestor, sibling, and descendant nodes, as shown in Figure 4), get more succinet but less noisy version of the observation. This version would be used for constructing the agent's ' focused working history.
review summary task) or the agent could interact with them to
B.2 PROMPT, PIVOTAL NODE SECTION
Generate the response the following format:
OBSERVATION HIGHLIGHT:
you would issue your action. Also include elements on the current wehpage would attend to if you fail in the future and have to restore to this step. Don't t include elements from previous higher hicrarchical level if their children nodes are considered crucial. Sort by relevance and potential values from high low. and separate E.g., '1321. 52, 838
of elements on current webpage numerical
pages. Seleet elements
B.3 EXAMPLE TASK OBJECTIVE
What the email address of the Dean of the School Engineering at Stanford University?
B.4 AGENTOCCAM 's WORKFLOW REGARDING THE PIVOTAL NODES
0. The task started at the Google.com, with the observation to be:
RootWebArea "Google"
link [29] "About"
link [30] "Store"
link [277] 'Gmail"
link [275] 'Search for Images'
button [282] GGoole apps"
link [152] "Siga
Iframe Presetttinnn [153]
[6]
combobox [12] 'Search' [required: False]
button [294] 'Search by voice
button [295] 'Search by image"
button [272] 'Google Scarch
button [273]
Feeling Lucky"
contentinfo
[83] "Advertising"
link [84] Business"
link [85] 'How Search works
link "Our third decade climate action: join us"
link [93] "Privacy"
[94] 'Terms"
button [100] "Settings"
generic [102] 'Settings"
1. The agent typed the keyword into the search box and identified element 12 (combobox [12] 'Search' [required: Falsel) and 272 (button [272] 'Google Search') as the pivotal nodes. The web transit Google search page with the searched entries listed. Now, the agent would be prompted
15

---

Published as a conference paper at ICLR 2025
with task history and web observation for issuing the next action, where the task history structed based on the pivotal nodes selected by the agent previously, and any pootallnode-associatee nodes. We take the automatically-genee -egenrated interaction history clip from the prompt this step:
<step_ -interaction
OBSERVATION: RootWebArea "Gongle" search
combobox "carrch' [required: False]
button "Search by voice"
button "Search by image"
button "Google Search" bullon "I'm Feeling Lucky"
REASON FOR ACTION:
To progress towards finding the email address of the Dean of the School of Engincering Stanford University. the first logical stop is to search for relevant information. Using the
input bex enter query such University email" and then submitting it using the "Google Search button is a direct approach to gather this information.
School Engineering Stanford
ACTION:
type [12] [Dean the School af Engineering Stanford University email] [1] 1sseep00.interaction
We can observe that the "pivotal node" technique helps keep a focused memory attention by only retaining the context of crucial information and omitting irrelevant noise. Based the interaction history summary and current observation, the agent executed the following steps, with the history section constantly updated with the information from the ongoing steps till trajectory ended.
B.5 SELECTIVELY REPLAYING ENTIRE PAGES WITH THE PLANNING TREE (FIGURE STEP 3)
We will use planning tree generated by our agent during the development stage as example for explaining how the planning tree takes shape and helps in selectively replaying past pages.
B.5.1 INTUITION AND BACKGROUND
If we make an analogy of the single-thread web task completion to code execution, the agent's issuing sub plans mimic nested function calling, the active plan is like the uppermost function the execution stack, and pruning the planning tree plays as popping failed functions off the stack. There constantly exists planning tree and the planning tree operation instructions in the agent's prompt. We enable the agent to organize the planning tree with 'branch' and 'prune" commands, where the 'branch' action creates new subplans, and the 'prune' action restores task progress to a previous state. It's noteworthy that those planning commands are in the same position as navigation prompts for the agent.
B.5.2 PROMPT, PLANNING TREE SECTION
If you think you should refine plan, use following actions:
branch parent-plan_id] nnw.subbllan.itteni:: To create new subplan based PREVIOUS PLANS. Ensure new subplan connected
appropriate parent plan by using its E.g 'branch [12] [Navigate to the "Issue" page check all the issues.]
prune [resume | [reason To return to previous plan state when the current plan E.g., 'prune
deemed impractical. Enter the ID of the plan state you
The current page lacks items "black speaker," prompting return to the initial page to search.
restart
Otherwise. use the following actions: cravigationsspecifiatioo.
B.6 EXAMPLE TASK OBJECTIVE
(WebArena 174) Open my latest updated issue that has keyword "feature" its title to check if closed.
B.7 AGENTÖCCAM WORKFLOW REGARDING THE PLANNING TREE
O. At the beginning, we used the task objective the root plan:
16

---

Published as a conference paper at ICLR 2025
(Active Plan) Find solution "Open my updated issue keyward "feature in its title to check if it is closed"
1. The agent added a subplan to planning node 0 by issuing branch [0] [Navigate to the Issues page to search for the latest issue with the keyword "feature" in the title.]. Now the planning tree changed into:
Find the solution to check if it is closed"
"Open my latest updated issue that has keyword "feature" its title
[1I (Active Plan) Navigute to the Issues page seurch for latest issue with the keyword "feature" in the title.
2. The agent navigated to the project's s issue page.
3. The agent decomposed the plan by generating branch actions branch [1] [Search for the latest issue with the keyword " "feature" in the title and check if it is closed.]
and branch [1] [Open the latest issue with the
keyword " "feature" in the title.]. It performed navigation steps for each active plan
before the next planning command was executed. In this example, after branch [1] [Search for the latest issue with the keyword "feature" in the title and
check if it is closed.] was proposed, the active plan turned into "Search for the latest issue with the keyword 'feature' in the title and check if it is closed." The agent then typed the keyword "feature" into the search box and sorted the issues by operating the sort icon before generating the next plan "Open the latest issue with the keyword 'feature" in the title." In other words, all the navigation commands (e.g., search and sort) it issued were intended for the current active plan (e.g., search for the latest issue with the keyword "feature" in the title and check if is closed). Just like function call only needs to consider the function's 's scope, this allows the agent to only attend the navigation actions dedicated to the active plan and the corresponding web observation as the playing history, which helps selectively replay past pages for the agent. Note that in this case, it assigned the two new sub plans to the same parent plan [1]. which automatically shaped the planning tree's structure. Finally, the planning tree reformed into (the content enclosed in "IJ" means comments, which is intended for illustration and didn't It appear in the agent's prompt):
[0] Find the solution to "Open my latest updated issue that has keyword "feature" its title check if it is closed"
[1] Navigate to the Issues page to search for the latest issue with the keyword feature" " in the title. [Plan 's action scape: navigate.]
[2] Search for the latest issue with the keyword "feature" in the title and check if it is closed. Pllnn's
scope: search and sort.] [3] (Active Plan) Open the latest issue with the keyword "feature" in the title.
4. The agent executed the following steps to complete the task.
C AGENTOCCAM'S GENERALIZABILITY TO REAL-WORLD WEBSITES
Table 9: Performance comparison between Agent-E success rate (SR) and AgentOccam SR on tasks defined on real-world web environments from Web Web Voyager (He al., 2024).
Website (Task Number)
Agent-F. SR
AgentOccam SR
Allrecipes (4)
75.00%
50.00%
Amazon
0.00%
0.00%
Apple (7)
57.14%
28.57%
ArXiv (16)
50.00%
43.75%
BBC News (2)
0.00%
0.00%
Booking (2)
50.00%
100.00% %
Cambridge Dictionary (9)
55.56%
$8.89%
Coursera
50.00%
50.00%
ESPN (10)
40.00%
50.00%
Google Map
33.33%
44.44%
Google Search (16)
62.50%
81.25%
Huggingface (17)
17.65%
29.41
Wolfram Alpha (34)
73.53%
61.76%
Overall (129)
51.90%
54.30 %
Web Voyager benchmark (He et al., 2024) compiles web tasks based on 15 popular roal-world web- sites. It comprises tasks with two types of user questions: ones with "golden" answers that
17

---

Published as a conference paper at ICLR 2025
Table 10: The success rate (SR) of AGENTOCCAM's ablation study with models GPT-4-TURBO and GEMINI-1.5-FLASS on the WebArena's 's development set.
Agent
Model
SR (%)
Shopping
Shopping Admin
GitLab
Mup
Reddit
Multisite
(#190)
(#48)
(#41)
(#41)
(#29)
(#21)
(#10)
Vanilla
GPT-4-Turbo
14.2
16.7
12.2
14.6
17.2
9.5
10.0
4 Actions
GPT--Turbo
25.8
22.9
24.4
34.2
31,0
19.1
10.0
Above + x Scrolling
GPT-4-Turb
30.0
29.2
29.3
36.6
24.1
38.1
10.0
Above Obs Opt.
GPT-4-Turbo
34.7
37.5
34.2
22.0
41.4
57.1
10.0
Above + History
GPT---Tubo
36.6
44.8
AGENTOCCAM
GPT-4-Turbo
44.2
45.8
46.3
48.8
41.4
47.6
10.0
Vanilla
Gemini- 1.5-Flash
11.6
20.8
9.8
17.2
0.0
4. Actions
Gemini-1.5 1.5-Plash
23.2
29.2
22.0
29.3
13.8
19.1
10.0
Above X Screlling
Gemini- 1.5-Flash
24.2
33.3
24.4
22.0
27.6
14.3
0.0
Above + obs Opt.
Gemini- 1.5-Plash
31.7
20.7
Above + History
Gemini- 1.5-Flash
32.1
35.4
31.7
31.7
37.9
28.6
10.0
AGENTOCCAM
Gemini- 1.5-
33.7
37.5
34,2
36.6
37.9
28.6
0.0
definite and time-invariant, and ones with "possible" answers that are either open-ended with multi- ple potential answers related to real-time information. We use Web Voyager questions with golden answers to avoid subjective human evaluations. We acknowledge that other WebVoyager tasks might test more web agent skills, but since they are not defined static web pages, and thus their solu- tions would change over time, each new web agent's 's success rate would require human assessment of trajectories. Due to the subjectivity of human annotation (as evidenced by the high variance in main experiment's human annotations provided alongside the code"), ), these results aren't t compara- group of human annotators label all previous agents' trajectories
ble. We can't expect the for every new agent, so we regretfully omit other tasks. Additionally, we exclude GitHub tasks, as the site's anti-scraping measures frequently cause page loading timeouts. Additionally, due to these measures, GitHub limits interactable elements, which prevents web page proper functionality, and the IP address hosting the agent risk of being banned. In contrast, WebArena's 's simulated GitLab environment mimics GitHub, enabling to demonstrate AGENTO similar tasks using existing results. In a nutshell, we have 129 tasks from Web WeyVoyager with defi- nite golden answers across 13 real-world web environments, including Allrecipes, Amazon, Apple, ArXiv, BBC News, Booking, Cambridge Dictionary, Coursera, ESPN, Google Map, Google Search, Huggingface, and Wolfram Alpha.
performance on
Our baseline on Web Voyager is the concurrent work Agent-E (Abuelsaad et al., 2024), which intro- duces several architectural improvements like the Blameer-rrowser--ellcooorr agent architecture and the browser's s document object model selection. It achieves the previous SOTA on the full Voy- ager with the assessments done by humans. As they didn't 't report agent trajectory logs, replicate their work on the WebVoyages subset introduced the above paragraph, evaluated with the same definite-answer-basee hard-coded evaluators as ours. We each task once for both agents.
Based on the results Table 9, AGENTOCCAM performs better than Agent-E on Web Voyager's definit-annswer-subeed Specifically, both agents have their specificities. We find that Agent-E makes an edge on websites like Wolfram Alpha with more delicate plans issued by its "planner," and AGENTOCCAM outperforms on websites like Google Search and Hugging Face with more accurate task interpretation and information retrieval. The impressive results across 10+ websites and many types of tasks can show AGENTOCCAM' generalizability in the web environment.
D AGENTOCCAM'S GENERALIZABILITY TO BASE MODEL FAMILIES
We conduct the full set of ablation studies on WebArena development subset with GEMINI-1.5- FLASH (Google, 2024), model trained with different data from the GPT model family.
Due the cost constraints, we construct representative subset from the original 812 tasks in WebArena. Specifically, we sample one task from each task cluster instantiated with the same intent
We would like to thank Yuhao Cheng, Tong Li, Yuren Hao, Ye Wu, and Xiaoxuan Wang for helping assess the agent trajectories.
18

---

Published as a conference paper at ICLR 2025
Table 11: Action statistics for the ablation study of AGENTOCCAM'S components with models GPT-
4- 4-TURBO and GEMINI-1.5- 1.5-L-- FLASH on the WebArena's 's development set.
Exp.
Model
click hover
type press scroll new go_back
gato
note
stop go_home prune
Vanilla
GPr--Turbo
473
26
212
22
4
9
115
4. Actions
GPT-4-Turbo
1612
515
107
16
40
120
Above X Scralling
GPT-4-Turbo
1641
491
19
60
128
Above Cbs Opt.
GPT-4-Tubo
1539
437
21
82
137
1
Above History
GPT-4-Turbo
24
181
AGENTOCCAM
4PT4Turbbe
1110
261
123
40
183
3
9
6
+ Actions Gemini-L.S-Fish
Vanilla
103
0
7
5
0
9
Domini--.55-Faaaee
1390
509
176
28
31
130
11
Above X Scrolling Gemini-L.5-Flash
1258
542
23
53
134
22
Above Ubs Opt.
Demiini-1.5-Faan
144
Above + History
Gemini 1.5. 776
253
50
44
185
15
AGENTOCCAM
Gemini-1.5-Flat 942
185
27
Table 12: Average observation tokens per step of AGENTOCCAM'S components with models GPT- 4- 4-TURBO and 1EMINI-115-FLASS --5-S on the WebArena's 's development set. We use the GPT2 tokenizer from HUGGINGFACE (Radford al., 2019).
Exp
Model
All
Shopping
Shopping Admin
Gitl.ab
Map
Reddit
Multisite
Vamilla
GPT-4-Tubb
2202.5
2268.7
2421.6
2267.3
1882.3
2119.0
1715.8
Actions
GPT-4-Tube
1682.1
1496.7
2186.6
2168.3
953.3
1102.3
1261.8
Above x scrolling
GPT-4-Tube
3571.2
3120.3
$175.5
4234.9
1423.3
3079.8
1977.9
Above Obs Opt.
GPT-4-ubb
2669.8
1664.9
4274.6
2744.3
1930.8
2664.4
1370.6
Above + Hlatory
GPT-4-Turbo
3107.0
1745
4988.5
3679.2
788.0
2758.4
1856.9
AGENTÖCAA
GPT-4-Tube
2785.1
1500.8
$032.8
3032.3
645.6
3476.0
1233.9
Vanilla
Gemini-I.5 1.5-lashh
2303.4
2307.0
2537.5
2707.0
1849.7
1794.7
2037.6
+ Actions
Gemini- 1.5. 1.5.Flash
1713.9
1577.4
2112.3
2257.1
818.5
1542.8
1177.2
Aoove X Sorolling
Gemini- Flash
3219.6
3110.7
5234.0
3378.0
1040.7
3204.9
2021.1
Above Obe Opt.
Gemini-I.5-Flash 5-FFlssh
2814.9
1769.8
4632.9
3410.7
858.7
3203.5
1577.0
Above History
Gemini- 1.5-Flshh
3283.2
1720.7
5078.3
3657.6
734.2
$712.5
1112.4
AGENTÜCCAM
Gemini- Flash
2872.2
1639.6
4156.3
2519.3
688.7
6241.4
1267.7
template (e.g., "What is the top-((n)) best-selling product in {[year))", )", where "n" and "([year) }r would be replaced by instantiation tokens) in WebArena, forming a development set with 190 tasks". We use Gemini-1.5-flash for all experiments. We run each task once and restart the experiment if the WebArena simulator fails (i.e., login expires. Reddit post limit exceeds, map malfunctioning, etc.). The results and statistics are listed in Table 10, 11, 12, 13, with the performance from the GPT-4-turbo counterpart on the same development task set for comparison.
We can observe a similar trend with the Gemini model that each alignment component introduced by our paper contributes to the web agent's 's overall performance. To be specific, removing non-essential actions (L
encourages the agent to explore more actively with commands cl ick and type; disabling scrolling (Above + X Scrolling proves advantageous in tasks where key information is not presented on the first browser sight; simplifying web page elements (Above Obs Opt.) .) reduces the observation token number; selectively replaying web element in one page (Above + History) reduces the steps required to accomplish the task; and planning and se- lectively replay past pages (AGENTOCCAM) enables the agent self-organize task workflow and quickly restore to previous stage after several failed attempts. In conclusion, each alignment com- ponent proposed by AGENTŌCCAM proves beneficial the agent system and can be generalized to different base models.
We remove s op in the statistics for the vanilla WebArena agent as this action is excluded in their officially defined action space. However, their agent is allowed by code to generate op to end the trajectory.
"The task ids are: 3, 9, 13, 20, 22, 30, 35, 40, 42, 44, 46, 48, 52, 61, 65, 68, 71, 76, 78, 80, 86, 89, 94, 96,
97, 99, 104, 109, 116, 117, 118, 120, 125, 127, 131, 132, 138, 141, 148, 153, 156, 157, 158, 164, 171, 173,
180, 187, 188, 194, 204, 205, 212, 217, 219, 224, 225, 230, 234, 237, 239, 244, 250, 257, 258, 259, 262, 265,
271, 274, 281, 283, 285, 287, 289, 294, 298, 305, 311, 313, 316, 323, 324, 333, 337, 340, 345, 350, 354, 356.
357, 360, 367, 368, 370, 375, 376, 377, 382, 383, 384, 386, 388, 390, 395, 401, 406, 410, 413, 416, 419, 423,
425, 435, 440, 443, 446, 452, 457, 463, 468, 472, 476, 482, 490, 495, 499, 503, 508, 509, 518, 521, 525,
530, 535, 539, 546, 550, 555, 561, 566, 567, 571, 578, 580, 587, 594, 599, 604, 606, 613, 615, 624, 627, 634.
637, 644, 646, 653, 660, 663, 666, 669, 674, 677, 682, 687, 691, 698, 702, 706, 713, 715, 720, 730, 731, 738.
749, 754, 758, 762, 766, 770, 771, 772, 779, 785, 797, 803.
19

---

Published as a conference paper at ICLR 2025
Table 13: Average number of steps per task of AGENTOCCAM'S components with models GPT-4-
TURBO and GEMINI-1.5-FLASH on the WebArena's s development set.
Exp.
Model
Shopping
Shopping
Map
Reddit
Multssie
Vanilla
GPT-4-Turb
1.3
1.5
1.3
1.1
1.4
1.3
0.9
Actioon
GPT-4-Tube
3.0
2.9
Scrolling
GPT-4-Tubb
2.9
2.2
Above
Obe Opt.
GPT-4-Ture
2.7
2.4
2.7
3.4
2.7
2.4
2.7
History
GPT-4-Tube
AGENTÖCCAM
Turboe
2.1
1.7
2.2
2.4
2.2
2.0
3.0
Vanilla
Cemini-1.3-Flass
0.5
0.5
0.5
Actions
Gemini 1.5-Flash
2.8
3.2
28
2.5
2.4
Aoove X Brrolling
Gemini-1 Sonini-1.5-Flash
2.5
2.4
22
2.4
2.7
2.7
3.3
Opt.
1.5-Flash
2.4
Above + History
Gemini 5--alshh
1.6
1.6
1.7
1.7
1.4
1.5
1.7
AGENTOCCAM
Gemini-1 1.nini-1.5-Flash 1.5-Flahh
2.0
E EVALUATOR RECTIFICATIONS
E.1 RECTIFICATION CATEGORIZATION
We only modify the evaluator when it's deemed erroneous due the wrong task labels misuse of evaluating functions. When the task definition and corresponding evaluation metric match to some extent but might be misleading to ensure the slightest reasonable changes. We emphasize that we re-implement WehArena's base agent SteP's agent with the same web environment and modified evaluators TOCCAM for fair comparison. For example, we keep the evaluators of shopping tasks defined with template 163, requiring the agent "Draft an email to the shop owner via their contact us funct ion for coupon as [reason]" which doesn't 't explic- itly specify whether to submit the drafted email. However, the evaluator is defined assess the not submitted email. All capable LLM-based agents have tested, which have been aligned to be helpful, will for sure submit the email not directly prompted to behave in the way the evaluator desires, leading the email field to be blank and thus failing those tasks. Another example of this kind is the Reddit task asking the agent to find the most appropriate subreddit to post (task template 6100), where the assessment of appropriation is very subjective. In all those tasks, we follow the original evaluators, though their evaluation outcomes are arguably questionable.
still keep the original agents and even to human,
AGEN-
We categorize evaluator modifications into classes, namely label errors and improper evalu- ation function selection, raise representative examples for each class, and list all the changes made." 10
We find there exist evaluator definition errors and typos in the correct answers. In the later cases, the tasks always require exact matching, where any well-aligned LLM-agent would correct those typos in their generation. We thus rectify those errors:
Label errors:
i) Evaluator definitions contain errors. For example, in the Reddit task 584, the eval- uator would open up the wrong page for the evaluation. Another case in point is the shopping task 261, where the urlmmatch evaluator is constrained to identify- :
ing one correct url (<server misjudging the same page (of the identical content) with a different url (<server :770t> eeto/eeonnoiiiio html Pcat=60). Tasks fall in this category include: 261-264, 351-355, 584, 644, 679, 707-709, 786.
tii The answer contains typos or grammatical errors. For example the is car necessary NYC in task 601, or the budge in task 603. More tasks of this kind include: task id 240, 298-302, 489, 583, 601, 603, 606, 608, 629, 645-649.
Improper evaluation function selection: Evaluator problems are more obvious in this case with following types:
10 As the evaluator is programmed by the WebArena simulator to be revoked only once at the end of each trajectory, our statement of "our approach outperforms the baseline methods with the original evaluators" refers to setting all the rewards of the trajectories with modified evaluators to he 0, which can be verified with the reported trajectory logs.
20

---

Published as a conference paper at ICLR 2025
Use the exact_maten function that compares whether the answer given by human label-er and the answer returned by the agent is identically the same. Errors occur when the agent returns a full- complete answer, where the evaluators' labels cannot match. For example, in Reddit task 644 that requires the agent to post meeting notice with the meeting date, where the keyword match for such date is exactly the Dec 15th, where the evaluator would judge other answers like December 15th as incorrect, where we change the keyword matching to one that could match both Dec 15th and December 15th. (In other cases with single answer, we simply replace which for instance in task 254 it could match 41257850C0
form
exact -xact.match with fuzzy-mate with the agent's answer The phone number is 4125785000, or replace exact_mat with must ww..nnllude which for instance in task 363 it could match 778m with the agent's an- swer 778 m.) also demands that the answer should strictly include expressions like virtual meetup, where the agent might add other words in the vi virtual and meetup. In that sense, we also split the keyword virtual meetup into two separate keywords, i.c. virtual and meetup. Tasks of this kind include: task id 97, 118, 146, 178-182, 254, 293-297, 308-312, 330, 363-367, 415-418, 528-532, 640-649, 653-657.
ti) Use the poorly defined fuzzy-match function, that would view the answer returned qualified for the missing-free wwangg-foomeeeeee expresxpnession answer exploration process, or assess answers with more detailed answers as partially correct (reward-O). We thus shift our prompt for the fuzzy match function from: 'Help a teacher to grade the answer of a student given a question. Keep in mind that the student may use different phrasing or wording to answer the question. The goal is to evaluate whether the answer is semantically equivalent to the reference answer." to 'Help teacher to grade the answer of a student given question. Keep in mind that the student has executed the actions to get the answer. They are allowed to use different phrasing or wording to answer the question. The goal is to evaluate whether the key points the reference answer are included in the student' an- swer. We allow answers with additional information that doesn't 't contradict the reference answer and review them as fully (not partially) correct."
iii) Misuse the Fuzzy_match match function by splitting the keyword list for matching into list, where each of the keyword and the entire answer, would be evaluated as partially correct (reward=O). In other words, no answer would be assessed as the correct answer (even the gloden-standard answer itself) due to such evaluator function misuse. This could be inferred from the function and the evaluator's definition. Tasks of this type include: task id 16-20, In such tasks, simply merge the list of keywords into a string, concatenated with ; " For instance, for task 16, the previous fuzzy match field is ["driving: 2min", "walking: 1 16min "], and modify
to ("driving: 2min; walking: 16min"].
E.2 DETAILS
the eval refereeee answers fuzzy match
I' d'diiiin::
2min". waakingg 16mia"
ddiiing:: 2min; walking: 16min
t Task 17
ehe refereefe answers. fuzzy match
driving: 13min' "walking
I' driving: 13ma; walking: lh 35mmn
e Task 18
I' d'diiiing: 15min walking: match
[' ('driving: 15min; walking: 1h 47min']
Task
eval raae.rreeeeseennwwaae. fuzzy mateh
I'driving: 12min "walking: la 44min.
('driving: 12min; walking: lh 44min.
eas eval refferenceeeee answers fuzzy match
[' d'diiiin:: 13min" "walking. 18 45min' driving:
walking: 1h 45min '1
* Task 97
dаU aval. reference answers must include
['914km"
['914km [OR] 914 km]]
* Task 118
MAU program.
[f'rrl" "last" "Incstor" "required contents ": {'must include ['jaw bruxiem "mouth goard
21

---

Published as a conference paper at ICLR 2025
I(url "last" "locator"s ": "required.costeens ("must.include I'jaw".
guard '1)
* Task 146
eval. waal.refereace.nnwers must.include
['16x24"]
ehe www.rereeeee.asswer..... fueeyys...uuy.sss.. futey.match
1 ['16229'
**y eval. cec.ceee...D.ees..cc............
Yes
waal.reference.answeae fuzzy
Yes. closed 'I
* Task 179
oval.reference
match
1 Yee.. 1t is closed 'I
eny eval. refereene answers, exact exact match
No
refereefeeece fuzzy,
I'No, open
e Task 181
eny relf-reference answers. match
('No. is
#28 aval. rrfference answers. exact match
Yes
fuzzy
[ Yees closed ']
# Task 240
instantiation.c ,iit.. product_category
competitive swimwear
competative swimwear
eau inteat
doing market survey stop category
expensive competitive
1 am doing a market survey for one stop market, show me the most expeasive product from competative swimwear
наи eval rew.erecce.nnwwerr.. wnswers. www.wwwwwwwww............... eatch 4125785000
reference.anwwers.
['4125785000']
# Task 261 [[ [f"reference_ur 'hrtp:// localhost: elettroocc.................. 6
# Task 262 eval. [[ (erreereence.url
wlothin -uuhiin--hhsss-owwell hhgg himl
# Task 263
[[" [f'refernnce_ue 'hup:/// localhoss 7770/ -shesss--hsss--swettt him)Feat=143")
# Task 264 Hf ('rreerence.url hhtp:://ecaaoost office -products hta.:uutt..immmmatat1le1 himlFcat==18"")
* Task 293
refereeffnnc answeess...s.. exact_maich
zit clone xsh :/ git melis. lti, es emu edu convexegg/ http:///////uu.rr..essmmmm.m gil
the wall.reference.c answers must_include
clone ssh:ff metis.
Task 294
*4# refereefee catteenceeansweee.. answers exachmmachh
clone ash metis. ccc.m.ccd:222///ooveeeggg//////
real.reference.
mustinclude
[' gii clone ssh ssh:ff zit metis. Iti cmm. mmmu.ed::133 com.:dd::22//oooeeee/hhhppr
# Task 295
referenceeanseee wnwseess... exact_match
metis. Sestiis..tt Iti, git clone ssh :/ metis.
es emu cdu:2222/root /etemmemass..... git
tAU real.reference answers must include
cdu.edd:22221/ metaseg,
Task 296
ene eval. www..eefeeeceeeee waw.eeceeeeasesseee.e...ss.. exact_match
metiis.
edu:2222/
22

---

Published as a conference paper at ICLR 2025
gigit
metis... cm..mm.
V Task 297
"ral..rfereeceeanwwe
ash git metis. cam, yll:::2211yyll0001000000
ehe eval reference ailrence.answerre must_include
I' l'git clone ssh metis. .t...t..c....................... cw..mm. www.cmm............ :2u:2222/ (1t.2///yoo/221991nco
eny inteat-template
Show the most recent [[status]] } order
the
[[starus]] order page
Show the most rerent completed urder
Show the most recent completed order
thy inteat.template
Show the most recent [[status]) order [[starus]] order page
the
Show the most rerent cancelled order Show the most recent cancelled order
#*s intent.template
Show the most recent [[status]) order [[starus]] order page
Show the most recent peading order
Show the most recent pending arder
eag intent template
Show the most recent [[starus]] order [[status]] order page
Show the most recent processing order
Show the most recent processing order
dаи intent template
Show the most recent [[xtarus]] order
recent [[status] page
Show the most recent out of delivery arder
Show the recent of delivery order
dаи eval reference cettrence_nnswer. answers eatch
Shawn Allen
rwwereecee.asweereeeeee..eeeeeee must_include Shawn Allen ']
4 Task 309
refeeeence.anwwers
Grayson Wright eau eval.reference seference.answers. iustuicl [' G'aayson Wright
Tark 310
sаu aval. rele.rrrffeeeeennveea exact_match
tokudu
refereence.nswerrs. must_inelude
['tokuda' ']
4 Task 311
Erik Linder-Nor\ enn
eau eval www.reeneeeanneeeeeeeeeeeeeeseee annwers, instuie
Brik Linder-Nori 'en')
Tack 312
the eval. wwal.reference.answw sat.ee......ts......mmmmttt exact.match
Christopher Groskupf
referenccnn
must_inelude
[' "Christopher Groskopf ']
Task 330
cva..reffrencc.nnnvees.c mustind
["81.31 1'83.31 MAy www...teeenss.aa..........e.oo
83.31
Task 351 eval.or [[" referenceeurl". 'htp http:///ecalhhs:::77000 -gdmo wtt.--gaee..hhtttasss/////eeooe www.:rrodddt.....ooeeeeereeeeeee orderrpies
Task 352
23

---

Published as a conference paper at ICLR 2025
I( rereerencc.urll http:://ooocalass:777hhhuu aoo.sool......aoaaaaapparrrrrrl.
+ Task 353
instaatiation.dic pic. product.categorr competitive swimwear
competative swimwear
eav intent
List products from competitive products from competative swimwear category by ascending price
category ascending
eny eval.or
If" ('reference url 'htp http:///ecalhoss:770000 277p///coo...7///ootctiii clothing -os htgp://pwweeetataaaaaatttrrrrrr hmm//catellP@pprducc list orderaprice
ehe eval.or (( (""refereace.ur 'hap Rtt kitcher dese
Task 355 env eval.or (("eeferaceeuu 'hrip lttaleastt Kitteeeeeektttt....m html"cat- product. wed.dett.iiiii..... darrdirreess"
Task 363
eav eval. weal..refeeecceaneen answwars. extct.match
748a
reffreecceeeee.... answers.
1777mm [OR] 778 m'1
eay eval. reffrrefeee wnw.waawwaa..... raw annotation
748m
778
* Task 364
vval.referense.anwaaa
#ad eval raffeeence answers, mast inelude
(1m.7km (OR 1.7 km]]
#8u eval. weefrrenceeanseee exact_mate match
2. 2km
mast_inelude
1.2.2km (OR] km/]
* Task 366
answers
#as eval
instuid
[1.2kkm (OR 1.2 km']
*аu eval. refereence.nswwers. answers, eatch
1.4km
km
# Task 415
HAU exal.
[f"url" 'hitp h//lo//lhout :0 wailyweiirilllyyy-eeiii..g. htwp
'locator "; 'document,
querySelectort') "idd"notee--l taaaflleeeet querySelecter () dimeeiin hody V").
outerText"
required_cosiemtt "s ('esact.match '; "ddavepgreeeee "H]
[f'orl" 'htpp :ff aateecahhhs::02000000 #023/hytehtazz balllaze/alllllerriiii club/-/merge rerprerrrussss/40 'locator ": "document.
querySelector [id==notes--iiss outerText" conueredd.comeetts ": instuin d'aveppgeeeeee
JastEllmmmnchhiie query
dimeline-discusiinn -body
# Task 416
prol.arrggambbtm)
If" "; "http lacalhost :0wph///rt0022/ala1 lyprojoct/a lyprojje
"lacator 'document,
vuerySelecior('s "odiddinote--iiss list lastElleeeemm querySelecctor timeline-discusse -bedy (").
outerText" required__ontentts : ("exact_maich" 'Thank yll
": htip lacahhost 20033/ot:8023/a11 http///ltyp/////////o/ lyprojet wwwm//mmegee__....eomm.m.
hmmpm//mep...epusss/200000 "locator 'docrment.
querySelectort) [id="notes-iis list "(" LasiEle query () timeline -idelinneddsssssoon -bddy \
outerText "required_costemts :; incluue ": ['Thank you }}
sAu eval. programmhtmm
H. url "; 'http :/ http::r:::000//aal/oooottttaaaa
com/ /ot.://mgg.....eussss/eee.eeee 'locator :: 'document.
vuerySelectorr's "id=="otex-ill Last.aalElemmm query -imelline-uixeussio -bedy \").
"required_costents
(eeaett ('esact_match "Rooharjossey" }}]]
(f'url is 'htpp loca/host: httppooote/allyypeeeee.e
stm///////eeeee hmmm-//mrgerrqussts/3553. 'locator : 'Jocument,
querySelecterr [id==notes- ((") IastElementChill querySelectorf dimeline-diseusee -bedy
required_costents ts incus. Roshanjossey (I))
Task 418
ene eval. progrrmm.taa
url "htip z/ 3023/host::023//yeeee
"locator ':
"ducument.
ducument.querySeleeee querySelector(
coverr-stauus lasttidi
texiContenc "required.com conteeetsddontttsss.eeeeeeee.e ("esact.match Baay }]]
[['url ": "http 2/ 8023/
"locator :
"dccueett document. querySelector( (oalleettorr..ccccce... cover-title "). textoooonn.
sqqirred_centee :: missiiideldde" :: ("Busy' }
**# iestantiation.dae diet heading
No privacy policy needed is this dystopian world
privacy policy needed in dystopian world
24

---

Published as a conference paper at ICLR 2025
Change page title "Privacy Policy" page on my site "No privacy policy
dystopian
world"
Change the title of "Privacy Policy" puge on site to "No privacy policy is needed in this dystopian world".
prll--rogrambbtal
If url :
eage/ wwtpe 'locator 'document querySelecter(( iippul name-" title "\ ".," v.vuu "required.contesss ": ("exact.match"; "; "No privacy policy is needed is this
"htip :/f
dystopian "1]1
If heep
patt://p//////appapp/se watpe//dd//paee.iffff/ "locator "document querySelector() 'iagat] name- title " "), value" "sequired_conteets ": ('exact_match"; "No privacy policy is needed in this
dystopian world ")]1
the eval aal--rogram.ber html
[(url "last" ", "locator" "; "document. querySelector()) "[ritte""Waat your mind ! Valaee
requized contents {'must iaclude I'refund' broke after three of use'. '009000130',
'12,99"]}]1
[('ul] "; "last" "locator" ddacument. querySelector() "[titte" "WWat's on your mind?") "I.valse". required contents :: { must include ": " reefund "broke" :. "three days of use "000000100,, 1
eny eval prl-prra html
[['url" 1; "last" ", "locstor" 7;
your mind?") value" i'must_iaclude" I'refund' broke after three days of use'. "089000148',
"document.
queeueeet..geeySllleee querySelector()) "[rirte-
required '169.95 (1))
[['ud ": "last" 'locstor" : ddocument. qperyyeetrory [[titte"WWar'' on your mind?") "J.value'.. conteet ::
i'muuu ": ('refund" "broke" "three days of use 000000148".. (1699951))
eny eval. pall-rrgaammmm.. html
[['rll" is "last ", "locstor" 1: "document, queryytt..eerreee... querySelactorr "[ritte="Waat your mind?") valval
requised.contest "68.88"13]1
('must_iaclude" [ ['refund broke after three days of use', '080000161',
[[ "; "last" "locator" ddccument, querySeletrrrrrrrrrr "[ritte-"Wart "Whar's on your mind?"),
"). "j.value"
required_contents ::
musst_iaclude" "; ("refund' "broke", "three days of use", "00RI0D1G"", 168.881))]
ea8 eval- pal--rrrammmra hrml
[['url"s "; "Tast" ", "locstor" " "document, queume querySelactorr "[ittle="War" your mind?")
value" valuue
required
{" ('must_include" ('refund' 'it broke after three days of "088000180', '512
If ('rrl" "last" ", "locator" "deumment querenttt-eeySlleeeel querySalnetor(( "Tittt="Whar's on your mind?"), (") value"'
required_contents "
mmmust_icclude" "; ("refundd "broke" "three days of use". "00R1001SO", 1112,99'331
eas eval. proo-prraammbaa
[f'url" "; "last ", 'locator" ";
"document, queryy Selector), "[iitle="Waat" your mind?") ("). ("must_include ('refund' 'it brake three days of '008000130',
required '1.63'1)31
[f'url" "; "last" "locator"s i; "ducument query Selettrr "Whate on your mind?"), "). value
required "s iimustiincldde" [" ("refund "broke" "three days of use", "008000150", 11,63133]
Task
eas eval. program_htm.
(''rl" "hitp :/f
", "locator " "document. Juccment..uerrySeleeee SuerySel
forum_deserippion ptw.t/t
value "required cootentts ( ("must_include"
"locstar C' parents plan lovers 11), ('url
"s hitp :/ lacs
sresment-gerryeelatte forum.sidebar ")..value 'required_contents "; ("must_nclluee" ": ["Cat friendly " 'Local vendurs".
Promotion "Toiic plants! []
'locator 'document. queryyeerrrrrrrerttrrr" deser-esctipttion "). value" required_contente ('must_include" ['Cal parents & plant levers 1)),
http 1tca/lo1t:9999/110009 Pta99///PPaasforCaaaaottsssssss
arl i hiip :/f Lacalhost:
Platts
"locator "; dlcement-qqeryy querySelectoor
forum.sidebar" value 'required "; ('must.include" CaCat friendly 'Local vendors
Promotion Tooii plants!"]]]
# Task 584
MAU program.html
'hiip 1//achhoo /9999////Krnokkk
": 'document, querySelectar ["Efommm_eescriptiin value"
required_centeet ': ('must.iaclude P'Paace for Karzoke lovers 11)), (. ('url': "hitp 1
Karnoke' " 'locator's '; "document, querySelecior ("d ("ooommmssidebaa") value "required_contents
must.include" !: d'devices ', 'setup 1
'locator 'document, query (errrm.eeesrrpttio ['Place for Karaoke lovers "1 ('url "hilp :// localhost :9999/ Kara/KKaaaee//aeee edit "locator ": "document. querySelector ("#forem.sidebal "). value" "required.contents
hhip 1otallost:9999///1 :99r/////k//ee
value rrqquired.com contentss ': ("most_inelude
("mustinclluee [''ddidei ('devices" "setup }}
Task 601
HAU eval. program.btm
[('rrl"
..lastuurl... "y", "locator" "document, querySelectorf s.abmission...iner outerText" "cequired.costents ": ("musst. ('must_include I'is necessary NYC (f'url "June:
...ast.url... ")", "locator "docament, querySelection (ector("" (".oorr..summan submission...inee "). outerText". "rqquired.costents" ": (nclut. ": I'is car necessary in NYC [OR] is a car necessary in NYC1))
Task 603
the instantiation diet. question
safe and budge apartment live aye budget apartment
ayc
the intent
Post question, "safe and budge apartment live is nye". in subreddit where I'm likely answer
Post question. "safe and budget apartment live in aye", subreddit where I'm likely get
program.htmi
25

---

Published as a conference paper at ICLR 2025
If ("url"s "; "fune:reddia redd:r.t...........ss.r// ...ast.url_.. ")%, "locator" "eecumentgaerryeiieee querySelector waw.sstrssaimmiii..........
outerText required.costents :: ['must.inclode ": I'safe and budge apartment to live 1ye H
Hf url "June: outerText", "reuuired.costenss ("must.include"
"funerreedii red.ir.............ll. .last.ur.... ")", "locator "; "document. querySeeeeee submission...inas
budget apartment live 11]]
Task 606 ene eval.
Hf'rl" reddit get post. ari(" last.url "locator" "document. queryyelector submission inner
outerText", sequired..cstents ("must.inctudd I' necessary '1) [['rl" "; "fune: wew.didd.g.....oo...... wew.rreeeege...os..eet.. J.last.url... "y", "locator "docament. querySelectort submission..inaer"
outerText". required contents ("must include" ": I'is car necessary |OR] is car necessary '1)
the instantiation diet question safe and budge apartment live
and budget apartment
Find a subreddit focused on topics related to city lives in DMV area. and post my question. "safe and budge and post my question, "safe budget
apartment live" there
Find cubreddit focused topies related city lives live" ehy exal- progrrr html
spartment
[('rrl" i; "fune: "fune: reddit-get.poo pedii--er.postteri ".....rr lost.url.... ")", "locator" "; "dacument querySelector( ("ooor(".sss include l'safe querySelectorrr "dccumeat.querySellee
submissioon..iaaa ").
outerText", "required_ costents
apartment to live "docament.
"fudc:reddit get post arl("
")", "locator"
suubiission
outerText", "seuuiedd.costets :: ("must.inciode". ['safe and budger apartment to live '13
Task
I[ [['ur] programm-baal
gedditgee,,ppe pett.post_rrle "adl""..ooo"" .last,urk url. "y", "locator" "; "document, querySelector submission. Janer onterText", "required costents "; {"must include is ["your opinian 'Fun thing to do is Pittsburgh "]}]
'; "fune: reddit,ger.post.tee Iaattt,......... urlst.ur... outerText", "required costents ": ("must include I' youour opinion Fus things to do in Pittshurgh }]
"locatar "dacsment, querySelector submission..iaaee
thing
Pittsburgh
Task
program eval-program
I[ [["rr]" "; "fune: "func:reddit gecdit.ger.post.erll "aall"".._ssstuuuu urlt. "y", "locator" ": "document, querySelector "ssioorsssssisiiii iisse
outerText", "required costents :: ("must mmustinnce include 1s ['book reading 'March 15th ', "virtual moetup }}
('url"; "; "fune: reddit.geeddddgg..e.ee. geddi.ger.post.erll Jaast,.... "locatar "dacament, query suhmission
outerText", required,coatenn "mmettp []]]]
i'must_inctude ["beok reading 'Mar [OR] March 15th
Task
program [['url" "; "fune: "func: reddit_get.post_erf "a.atl"s.....u ,.Inst_url_.. "y", "locator" "; "docament, querySelector submission__.iee iiner ").
outerText", "required_cosii comtentts :; immustinnlude ['Harry Potter 'July 8th ", "virtual moetup H
(('url": is "fune: reddit__et__ gerdditgerrpost.eer .r. "locator "dacament, quergyggeeeeeee suhmission__
outerText". required,coatent [
("must_include ": 'Harry Potter 'Jol 8th (OR] July 8th 'virtual
PAU eval. program html
[['url" "; "fune: "func: reddit.ger.post_erl. (...ssst iistturr__. 1)7, "locstor" "docament, query ( (".eeetorrr.slaaaa suhmission__.inee inner ").
outerText", "required.comient : ('must.includl ['Big litlle lies 'Sep 10th "virtual meetup
"fune: reddii.got..pott.ett ('....st..rrr....... "locator "; "document, query suhmissiion
onterText", coepeiedd_coatets ": ('must_include [ littee Hix 'Sep 10th [OR] September 10th
virtual ', 'meetup |]
PAU eval. program.btm)
[f'url"" ";
(..asst,...... _last_url_,. ')7, "loestor" "; "docament, querySelector ("), suhmission._.nee
outerText", 'required.coatent: : ('must.include" ['racing "Oet 21st 'viriual meetup
i; "fune: geddil getdii.eet.poot.eee .rr............ "locator "document, query Seleetele submissionn..in innerr
outerText", required_contenn ": i'muus_inellde ("racing ears "Oe (OR] Gctober 21x1". "virtnal
''meetup }}}
Task
eAl Instantiation. subreddi
games
Post notice on virtual meetup for Tears Kingdom enthusiasts on Dec 15th in the games subreddit
Post a notice on virtual meetup for Tears of Kingdom enthusiasts on Dec 15th in the gaming subreddis
the rell.reference.ur
hitp 1tta:///oooss 99099////0000//mmmmm
hitp :// localhost 99999/E/gaming
HAU eval. proggram.btm
[('url" "; "func:redditee" ceddit.get.postannl. ..Iast.url... "y", "locator "dacument, querySelectorf "tafscctorr..ssmssssiiio submission.. iiner
outerText" "required_costents"" ('mustt.i ('must_include T'Terrs Kingdom" 'Dec "virtual meetup 1)}]
[f'url ";
reddiit.et..post.err ..last.url... ")", "locator" "document, querySelectonn (" submission..inees
outerText" "rquired_costents"" ("must.include"" ": Terrss of Kingdom" "Des 15th [OR] December 15th
virtual 'meetup ]}]
Task 645
e*e intent templatem
Post {[subreddit]] subreddit about could machine learaing
correpang field.
{[subreddit].
learaing help corresponding field.
the intent
Post books subreddit about what could machine leazaing help the correpong field.
Post books subreddit about what could machine learaing help the cortesponding field.
programm..mmmm
26

---

Published as a conference paper at ICLR 2025
I T'url"; "; redit--redi..gtt..ooor "..sstt".....lll...... ")", "locator "dacament. sueemmennggcrySlccttorr sasaiiiccoo...iiiiiiooo.aaaa
outerText" ", required.costents ["must.inclode ": machine learaing 'help (1)pp1))]
Hf url "fun::reddd red.ii.......sss.rr last.uul... ")", "locator "; "documeni. querySelector( submission. inner ").
outerText". "reguirec.conteuts" ": ("must.inctude
machine learaing
[O8] benetit
improve JOR] enhance [08] transform (OR] revolutioaize 31
e Task 646
the template
[[subreddit)] subreddit what could midjourney help correpang
Post in {[subreddit]] subreddit about what could midjourney help the corresponding field.
thy intent
could midjoumny help correpong field.. subreddit about the eval program html
Post
subreddit
Post
midjourney help the correspoading
[[ "fune: reddit-get.post.erll post.arl((...ost..uu ..Iast.url... ">", "locator "dacument. "ddcumant.guerySeeeee querySelectorr "tt.sstttt.ssmmmsiiionn..t ").
outerText", "required contents ': ("must include" "locstor "; "dacument. SeerungggeyySeeeeeeee suhmissioo ..inerr
midjourney"" 'help" [1]]
[["rl" ['url" "fune: reedit-ger-port-srl. "a..stt""" J.......rrrl.....
outerText", "sqquired.costetss"" ": ("must-include" "; midjouraey ". 'heip [OK] assist [OR] benefit [OK]
improve [OR] exhance [OK] transform [OK] revelationize "I]]1
eay intent template
Post in [[subreddit)] forum about what could open-source LIMs help the carrepong field. open-source
Post [[ubredd:t)] forum
corresponding field.
Post in technology forum about what could open-source LLMs help the correpoug field.
Post technology forum about what could open-source LLM help the corresponding field.
program
[(url"" ": reddit ...stt. ")", "locstar "dacament, "dacmmant.qucrySeeee hww.Sslccooossammiiiioo...a
outerText", "required_costent contents ": ["must inssuti I' s'open-sourze LLMs : 'help "1]]
[(urr "url"" 1; "fune: reddit post arl(" lost url - ")", "locstor "; "dacament querySelector suhmission inner
outerText", "required,costent" ": ("must,include ('open-sourze LLMs". 'help [OR] (OR] benefit
OK] improve [OR] enhance [OK] transtorm (OK] revolutioaize 11}}1
t Task 648
templale
large language madels help carrepong field.. Post in [[ubredditt] forum about what could large language madels help the correspoading field. sau intent
[[subreddit)) form about
could large laaguage models Post dataisheautiful forum about could large laaguage models help the correspoading field.
dataisheautiful
the correpang field.
sau aval. program html
[("rll" "; "func: reddit.get.post.nii (".aar""""....uua arl., ")", "locator": "; "dacsment, dlecmmnnt-guerySeeee querySelector sahmission_..iner ").
outerText", required_costenn
f'muut-include "s ['large language models' "help ]]]
Hf "s "fune: reddit ger past arl(" last ")", "locstor". "dacument, queemeent-qeerySee querySelactor
submission inner
outerText". "sequired_costent" ": ("muss_include" ['large leaguage modals" "help [OR] assist [OR]
benefit (OR] improve (OR) enhance (OR] transform OR revolutionize }}}}]
dau iement.template
Post in {[subreddit]] subreddit about what could diffusion model help the correpong field. {(subreddit); subreddit what could eau intent
help corresponding field.
Post in history subreddit about what could diffesion model help the correpong field.
Post bistory subreddit about what could diffusion model help the corresponding field.
aval. pro..rrgggmmmhtm
[('rrl" " "fune: reddit..et..ost..rii ,,Inst, ')", "locstor "dacument, querySelcetor (taabacoor.saammsssion..ii ").
outerText", required_cosiemt ": include [ diffusion model" 'help H] ..Inst,url... ")", "locstor "; "dacument, querylleeryS (". sahmisssoon_
Frrl" is "fune: reddit.get.poot..ne
outerText" "eequired_costetts ": i'must.include difffssien improve (OR enhunce [OR] transform [OR] revolutionize }]]
'help (OR| [OR] benefit
# Task 653
eval. pro...pogram.btr
[[" url "; "last" "locstor" "document, -eremmett-ueryyeleetteee querySelectorr( "[iitle="Wat your mind value" requized_centenn ": iimclue ": ['refund' 'it broke after three days of use '. "000000100",
hacoment.querySeleer querySelactaar( "[iitee""Wat your mind?") required_contenn :: m'must-include" ['refund' brooke after "three days of use "000000100",
"last" "Incstor" "dacmment,
BO8TOIN9W1 1F1]
PAU aval program.htm]
H. url " "last" ", 'locator" "
'ducument. SeerySel) 'fittee "What's your mind?") () value"
required_.ontemts "
mmmssiiilulee ['refund' 'it broke after three days of use ' '161', BOPPBFLII
(''rl ": "last". 'locator" documentt querySelector() quermntt.uerySelecccee mustinncldee ": ('refund" "broke". "three days of use '161". "61". BASPTFLLU
"Titte= "What' 's on your mind?"
"). value
required :
ehe aval. program.htm)
[['uri" "; "last" ", 'locator" "dncument. Suermett.geeryyeeeetctc SeerySel) titttee a "Whr''s your mind?") (). value"
required_centents
musstiaclude ('zefund' 'it broke after three days of use " '180', "BOSTQIN9W1
H url llast 'locator" ddccmment qweryyyeetrryyyyrrrrrrr. ( "ititte= "What' on your mind?"
va.valse"
cenuined :
instlie :: ('zefund' "broke" "three days of use '180" BOSTOINWW
the eval. progpppormmaar
[['url 1g "last" ", "locator" "; "document, queryyyeetttyyyyrrrrrrr 'I tittte "What' your mind (), value.
required.contents musssiaalude ": I'zefund" broks after three days of '180', BB00IMSF25
1>>1
27

---

Published as a conference paper at ICLR 2025
I( ["url"s "last" "locator" suerysletortyy "Ititle="War" your mind VJI.vauee i requized.contents :: must.iaciude refundd "broke" "three days of use "180". B00NIMSP25 1)
Task 657
eval. pra.-poggrammtm
[f'rr"" "last locatur" :: "document,
queryySlee. querySelrctor() "[title="Wart is on your mind?" ").
required.contentt ::
{ must.iaclude I I'refund" "broke after three days of use "148", "MOA ]
H''rl" '; "last" "locator dducumentt querySelectorr) "tittee "ittl="What'
mind (n (. value
required.coateass
musst.iallude refundd "broke", "three days use "148". "BOEVWSWAA
* Task 679
wall-program.bral
H'url"s : "last" 'locator" "dacumett.gerryeeeeeee. dww.roodddd..ddmn.dddm. adwii..daia-gidd-iiiid "uurret""
"dacument. querySelector
required contents : {' f'must Completed }]
[('rd : "last" :, "locator" : "document, querySelactor(" (ercctrr"divv div. wwii.admin...... waw.addd.daaaappppppp........ -ridd-iiltees "current"". osterText required
('must iaclude I'Complete 11)
t Task 707
ene eval- prog.pr html
[["rrl" "last" "loestor "document, www.maeenttggggSllle.. guerySalactorr "idd="saees.eeprrt (\am") value" }. ('ull H.
"required.conten"
["exact.maachh :; 11/1/2022
", 'locater "documeat. Sueepeaetrgerrreeeerrrrrr "[idd""""""""""""""" "iid==sales.ccpart..
3) ".,uuu'"' - "required contents :: ('exact match ": 112/31/2022"11
[(urr" 1 "last" "locator" "; "document, queemenn..nerySellee querySelactore
"iid=" wet.reettt.tammt "\ value" 'required required.coatents contentss
('exact "1/1/22)), ("url "last"
"document. querySellctoor() "lid="sales report, V)).value" "required contents ('exact match '12/31/22" }}
e Task 708
proggra
Hf ['url": "last", "locator "dacmment,
Sacmment.guerySSleeeee. querySelactor(()
"\". "). value required.coatents
["exact match : "1/1/2025 }. ('url'; "las:" "locater" "; "decument, querySelectorr "[id=" report
".llv " "required conrents "; ('exaet match "; "12/31/2023"
[('ur' "last" "loestor" "docnment. wew.uenn.nnoryeleeeeee.....
"aldd"
xaxlee 'required,
("exact match : "1/1233)) J. ("url" "last". "locator" "document, Seeement..eerySellee. querySelectorr) "lid="sales report
V...vluu' "required seaiireddcone contents ": ('exaet match " "12/31/233 []]
Task program
[['rl] "last" ", "locater", 'dacument, querySelector()) "add="saesssssseppee froott,fram" fromm) "). 'required,conte contents ";
("esact match " "5/1/2211 }. ("url ' "Tast ", "locater "; "document, querySelector "[id=" sales. report to ("exaet_match
"...valee" "reguired_conteats secemeen.geerySeeettee. ("exaet match "; "5/1211" H H. ("url" "; "last ", "locstor" document, querySeleeter() "[id=" sales report to V..valuu' "reguired_eonteats " ("exaet_match" ": (3/31/22
'3/31/2022"
[('ud' 1 "last", "locator
"Tid="sales, rerarreepttttttte vallaee required,coatente ';
eas eval eval. rewe..eefeeeeennnee wnswerss.........ee............. instuie ['412"]
[1414"]
Table 14: Action statistics.
Exp:
click
hover
type
scroll
back
goto
stop
go home branch prune
AGENTÜCCAM
197
769
42
47
AGENTÖCCAM SteP
5235
198
1407
11
25
132
124
1733
AGENTOCCAM Judge
726
Table 15: Average number of steps per task across all WebArena sites.
Exp.
All
Shopping 6.7
Shopping Admin
GitLab
Map
Reddit
Multisite
AGENTÖCCAM
9.0
9.2
10.8
8.5
8.6
13.3
AGENTOCCAM SteP
10.3
12.0
14.6
AGENTÖCCAM Judge
9.4
6.7
10.5
10.6
9.6
8.4
13.5
Table 16: Average observation tokens per step across WebArena sites.
Exp.
All
Shopping 1634.2
Shopping Admin
GiLLab
Map
Reddit
Multiite
AGENTOCCAM
2932.1
4920.7
3126.8
1056.0
3697.8
1282.9
AGENTOCCAM + SteP
2601.1
1675.2
3833.3
2983.8
1196.4
3071.4
1581.9
AGENTUCCAM Judge
3285.4
F ADDITIONAL EXPERIMENT DETAILS
We include the trial statistics for experiments that combine AGENTOCCAM with other compound agent policies like SteP's strategies and newly proposed Judge role. Specifically, 14 shows these well performing agent are equally open to web environment exploration, actively issuing environment-changing actions like el ck and type. Not surprisingly, the AGENTOCCAM SteP
28

---

Published as a conference paper at ICLR 2025
Table 17: The success rate (SR) AGENTOCCAM's ablation study on WebArena.
Agent
Model
SR (%)
Shopping
Shopping Admin
GitLab
Map
Reddit
Multisite
(#812)
(#187)
(#182)
(#180)
(#109)
(#106)
(#48)
Vanilla
GPT-4-Turbo
15.9
10.0
22.9
21.7
16.7
4. Actions
GPT-4-Turbo
25.9
23.5
23.6
24.4
34,9
33.0
12.5
Above X Scrolling
GPT-4-Turbo
31.7
26.2
25.3
35.0
33.0
52.8
14.6
Above + Obs opt.
GPr-4-Turb
37.4
26.7
45.0
16.7
Above + History
GPT-4-Turbo
38.2
33.7
40.1
31.7
50.5
$1.9
14.6
AGENTOCCAM
GPT-4-Turbo
45.6
37.8
61.3
14.6
agent frequently issuing un-interactive actions like hover. From Table 15, we can observe that AGENTOCCAM finish the task with the fewest steps, often yielding a task result with 9 steps. Last, from Table 16, those three agents' token consumptions are of comparative orders of magnitude.
(a) Shopping.
Shupping admin.
(c) GitLab.
(d) Map.
(e) Reddit.
(f) Multisite.
Figure 6: Success patterns of AGENTOCCAM (leftmost in each sub figure), AGENTOCCAM SteP, and AGENTOCCAM Judge (rightmost) across different sites on WebArena. The y-axis represents task ids, with green indicating successful trials and grey indicating unsuccessful trials. Notably, tasks defined with the same templates are clustered together.
As shown in Figure 6, agents that combing AGENTOCCAM with compound agent policies possess different behavioral success patterns. For AGENTOCCAM + SteP, it benefits in tasks where the agent could easily be guided with detailed instructions. such as shopping tasks, with more success (green) blocks and denser success rate in tasks defined with the identical templates. However, it falls short tasks that require generalizable skills like shopping admin tasks, and in tasks where task specifci strategies distract, like reddit tasks. On the contrary, AGENTOCCAM Judge agent shares similar patterns with the AGENTOCCAM agent except that some the success blocks are denser, thanks to the behavior rectification enabled by the action generation and selection pipeline.
In addition, we add the success rate figures of the ablation studies Table 17, which has been vi- sually represented in Figure 5. During development, slightly modifies AGENTOCCAM's prompt such as improving the wording or correcting the typos of the prompts, which don't affect the se- mantic meanings of the prompts add any additional information, and are reflected in the reported trajectory logs. As some failed trajectories are induced by the invalid interaction, we improve the interaction scripts, though not perfectly it would be beyond the scope of this paper, with the following code shifts:
# In brawser eSN
def execute_ action
setion:
page: Page
bro:ser,etx: BrowserContext.
obseration processor: OheervationProeessorr
Page:
match actios.1ype:
case ActionTypes CLICK:
each kind in order
shuyarrh) order temp now
29

---

Published as a conference paper at ICLR 2025
action |" "'elmment.id"
node obseration.prueessec get. Sec.aeee.sfo..byelemee.e. (itteelmmmmt)d)))))
molu=="mmuiiee if node and node. role== menuittm" and node. parent and node podee.aremt rareat.. node (node.
mmenuitmm" rale.rool=""ptte "opttoon
try:
page. gel.by.ee....eee..ee... rule (node.rule name=node name, exact=True), click) except:
page.get-by.roo (aode (ande..rle
came).clicke
except:
try:
page. gee.gger.by.role node.
select.option (node. name)
except:
page cet.by-role (node. parent. role. name=node pareat. w.t.:n select.otton node. name)
not obseration obseration.procestt processor element.ia, visible (page. element-id):
else:
page. get.by.te....ee wow.eyy--leeeeeeoo
exnet-Trme). click()
except Exception iry:
p"aantt click element rale and exact name. " bage.get.by except Exception
click()
Caannatt click by element role and fuzzy name.",
element. action] "lement_id"'
element
obseration processor. get | [
(clement id page) rype
execute mouse elicklelement centereeeserrrreee [0l, element center|]] page)
elif (aeoon""eemmmn role"] and action [ element same "]:
clement,role int(action relemeat_rale"" element, name actionl element name"
ath action ["ath"]
execute, fossteefocusseee (elementtroll
clemeat,name, nth, click (page)
action p" code"]:
paa.epallwiigs..ooeeeeee ("p.tooo"""""o"""""" loestor code parsed code[:-I]
parsed_code
[shayanzh] don't support args and page=page)
execute, Sacute_playwrigg. www.yaaiggthelliee.... wow.oaatoooooeeooattooooooo
clse:
raise ValucError
locator found for click action")
case AcpionTypes. TYPE:
if action (" "lement_id"1]
if not obseration. obseration.pracee ptt.tiiooooooostrrrr.... element.i iineentiis,,,eee visible element,id): prest_enter True _id2key [action [nction
[[x]][[-]]] "\n" alse
sode ibscration.processun get_ by (lememm..)))))))))
try:
press_enter:
page. guge.get.by.role idx in action[" [[:
name, exaci=True). (*". join Lidkkey [idx]
time. sleep())
keyboard. ("eter"E)))r"))
p by, role ralee rale, name-aode name, exact-True). fill (*", join (idkke [idx] [idx] for ids in action[" text []]x]
bet,by,
press_enter::
page.get get.by role (mude. role, nome=node name)) fill j"*,join ([ (L.id2key [idx] for action ["rexi []] -[:-]]))
time. (lrep())))))
page. keykaarb press Enter")
got.by. zolo (aude. role. nome-node name-noee.amme..... fill ("".join Lid2key [idx] for ids in action [etext" (I))
element.id = action clement_id")
element_center obseration.pracessee
ptw..llmm..ccttrrrrremm.... page) type: ignore
aattr deeinedd]
execute_mouse.ut mow.ee.eooeeeliekeee wtt.:iiiiiemmmiieeeteee.e. [0]. element..cntte((]] [1)), page)
page. keyboard press Control+A")
for in range(1):
keybbard. press page
pressdd..ppssssaccccaa execute_type "ettion"text"""" page)
clif action! "eeimen"elemmnt clement.role"" and action element.same"
clement.role intiaction "elemmsttrolle clement.name action "clement.name"
ath action ("nth"
execate.foce cereffocusselemme elemmstteem pype actien|"
clemeat.name, mth. page)
action petion""pw.code""
parsed.code
locator.code parsed code::ccde::--
[shayanzh support args kwargs
execute-playwe waaaywwightttype...
text=text. lecator.ode--icctoo..ood page=page
30

---

Published as a conference paper at ICLR 2025
NotiuppeementedErroc
"No proper locator found for type action"
G
AGENT PROMPTS
G.1 AGENTOCCAM
The general prompt template:
With planning
are assistant performing tasks hrowser. provided objective current step web page observations, previous plaas and interaction history. You need to issue an action for this step
Generate response
following format:
fourput specifications)
You ONLY
following
Strictly adheres
Only
single action.
If you think you should refine the plan, use the following actions:
{plansing actios. specifications?
Otherwise. ceaatiinnspccfiiiioo specificattins) ceavigation,actionn
follawing actions:
Without planning
will provided objective step page observations, other relevant information. issue action for this step.
assistant performing
Generate the respanse in the following format: {ourput specifieations)
You are ONLY allowed to use the following action commands. Strietly adheres to the given format. Only issue one single action. {navigation, _attion,eetoooooooooo specifisatiins)
Output specifications:
Interaction history summary: Emphasize all important details in the INTERACTION HISTORY section.. Observation description: Deserihe
the CURRENT features that are relevant ur patentially helpful fulfilling the ubjective detail.
Emphasize elements
Resson: Provide your rationale for proposing the subsequest action commands here.
Action: Select your actioa here.
Observation Highlight:
webpage current webpage you would attend which you would you fail future and have to restere to this step Dun' include elements from the previous pages. Select elements higher hierarchical level if most their children nodes considered crucial. Sort by relevance separate
include elements elements
iasue your action.
potential
high
E.g.. 11321,
Action space specifications:
Planning action specifications
branch [parent plan id] [new subplan intent To create subplan based PREVIOUS PLANS. Ensare the new appropriate parent by using the issues prune [resume plan id] [reason]] J: To return to previous plan state when the current plan is deemed impractical. Enter the ID of the plan state want resume. E.g.. prrnne [5] [The eurrent lacks .]"
subplaa
"branch (Novigate
"Issne" page
items black speaker," prompting
initial page
the
Navigation action specífications
click [id]: To click on element with irs numerical ID the wehpage. E.g., 'click [7] If clicking os your desired web this cases, move interact
lack interactivity trigger the relevant elements INSTEAD.
specific
the element similar or
visibility.
[id] [content] presss, enter_after-Ol HE To content into field with specific ID. default. the 'press
E. 'type [15] [Carnegie what you're looking is your attempt, coaxider back: To reture the previously viewed
key is
typing
Mellon University] [I]' If you can'r
refining your search keywords by breaking them down or trying related terms.
later stop [answer]: To stop interaction and reture respanse. Present your answer within the brackets. If the task doesn 'I require a textual answer appears insurmonatable. indieste 'NIA" and additional reasons and relevant information you gather the go home: To reture
[cantent]:
impartant info
completing
enable
8.. note Spent
4/1/2024]
E.g., 'stop 47min]
homepage where you can other wehsites.
31

---

Published as a conference paper at ICLR 2025
Observation space example:
RooeWehAres [1]
"Dashboard / Magento Admsin
link Magento Admin Panel
menubar [85]
link [87] 'DASHROARTY
link [90] "SALES"
[96] 'CATALOG"
[102] "CUSTOMERS"
link [108] "MARKEING'
link [114] 'CONTENT"
RRFORRSS'
link [138] 'STORES'
link [144] 'SYSTEM'
link [150] 'FIND PARTNERS & EXTENSIONS
heading 'Dushboard'
link [254] 'admin"
lick [256]
textbox [894] [required: False]
"Scape."
button [262] 'All Store Views'
link [265] What is this
[240]
HemlerA [898] "Advanced Reporting"
text "Gain new insights and take command of your business" performance, using our dyaamic product order
link [902] disahled. enable chart,
Advanced Reporting"
text
link [906] "here"
text "Revenue"
text "Shipping"
text "Quantity"
tablist [57]
[59] 'The information this
changed. contains invalid
link [67] "The information in this tab has been changed. This tab contains invalid data
text "The information this been changed." text "This tab cuntains invalid data. Please resolve this before saving.
text "Loading....
[61] information this
changed.
invalid
link [69] "The information in this tab has been changed. This tab contains invalid
text "The information this been changed." text "This tab contains invalid data. Please resolve this before saving.
"Loading....
[63] information
changed.
invalid
link 1711 "The information in this tab has been changed. This tab contains invalid
text "The information
been changed."
text "This tab contains invalid data. Please resoive this before saving.
"Loadiag....
tab [65] information
changed.
invalid
link [73] "The information in this tab has been changed. This tab contains invalid
text information
changed."
text "This tab contains invalid data. Please resolve this before saving."
text "Loadiag...
tabpancl information
been changed. This contains
table
row Product Price Quantity |'
row Sprite
row Quest Lumaflex Band 19.00 | 6 |'
row Sprite Yoga Strap 6 foot 14.00 6
Sprite Stasis Ball 23.00
row
Overaight
text 'Lifetime Sales
text 'Average Order
'Last Orders
row
Customer
Items
Total
row ---
Miller
194.40
row Grace Nguyen
row Mott Baker 3
151.40
Lily Potter
188.20
text 'Last Search Terms
table
row Search Term
Results
Uses |"
row tanks
32

---

Published as a conference paper at ICLR 2025
row
rOW
Joust Bag
rUW
bullister
19
Tank
23
text
'Top Search
table
rOW Search Term
Results
Uses
rOW bollister
1110
1OW Joust Hag
10
4
TOW Antonia Racer Tank
23
12
tanks
rOW
contentinfo
link [244]
"Copyright 2024
Commerce Inc. rights reserved."
"ver. 2.4.6"
link [247] "Privacy Policy'.
link [249] 'Account Activisy
link [251] RReport Issue"
G.2 JUDGE USED IN AGENTOCCAM + JUDGE EXPERIMENTS
The general prompt template:
You are se sensoned web navigator. You now assess the value and risk of serveral weh navigation actions based on the objective the previons inreraction history and the web's 's current state. Then, you seleet the objestive falfillment
action with reward the furars.
value and least
which you would
Adhere to the Following output format: {ourput ppecifications]
Note that "branch" and "prune" are planning actions that will modify the PREVIOUS PLAN section and won't interaer with the web environment.
Output specifications:
progress
Review critically why plans have
fulfilled objective nchieved.
Justiry your assessment with detailed evidence draws from the objective. observatioss. actions
Temiie the assessment using this format: -- plan If plana n[f
milestone H
[fconcrete proof from observation [[ milestone
successful H] [[eenerete.proef_ (ram_observation ]] (awy.mm)ssttttttost why.millestone.h steesssstt
happiids__akeefor.roo miiilttttt H
Action assessment: Asess the value and risk of each action. Consider both the best-ense and worst-ease
outcomes resalting from its implementation. Hemize the assessment ssing this format: i- action
outcomes you expect executing action id [action sote of the most correct and comprehensive content] [action risk including
including
limited
action. whether
but not limited to whether the note/stop content. is correct. and whether you can gather more information
by continuing playing rather than ending the trial] [[ best.ease ] worst_ease H]". your selected
Action selection: the
You only choose actios. E.g.,
H FUTURE DIRECTION: TRAINING EVOLVING WEB AGENTS
Our current alignment of the action and observation space is designed and deployed based hu- man heuristics under the assumption that the base models are not trainable. This approach excludes certain useful web actions web element formatting that could potentially be learned with minimal examples, rather than being mastered through training AGENTOCCAM, For instance, incorporating the scroll action could significantly reduce the observation's 's volume and better align with the design of real web environments, c.g., many shopping websites are designed to load more products only when a user scrolls down. Considering that the agent can record interaction traces deter- mine whether to scroll up down, it is likely that an LLM could learn this action without extensive training. By lifting the restriction on the non-trainability of LLMs, incorporating such design con- siderations might enhance the performance of web agents and reduce operational costs.
Furthermore, given the evolving nature of web layouts and human needs, developing adaptable agents is essential. Their autonomy is not only reflected by the ability adapt ever-so-updating web tasks, but also by the creation of more sophisticated observation and action mappings (functions and tools get them more familiar with the web environments regarding perception and action grounding, rather than relying on human heuristics, just as what has been done in this work. Such agents would be able to autonomously refine their skills and improve system performance.
33


Reviews:
[{"review_id": "UySUOf4zvM", "reviewer": "Reviewer_NaFf", "summary": "This work focuses on developing a simple generalized framework for LLM-based web agents, allowing them to leverage the strengths of LLMs to get complete web tasks. This framework focuses on aligning LLMs to the web domain and hence converts the problem to that of finding alignments/mappings in observation and action space between web domain and LLMs. As part of that, for action space alignment, they reduce actions to simple set of actions and add few additional \"planning\" related actions to the set. For observation space alignment, they propose to simplify the web page representation and also focus on selective representation of history for tracking. They show impressive results of the impact of these strategies on WebArena benchmark while also providing ablations to understand the impact of each of the recommended change. Finally, they show how this simple baseline agent can be promisingly combined with existing strategies.", "questions": "- I would be curious to see how the agent performs on other benchmarks, especially WebVoyager. This is especially valuable benchmark as besides being more realistic, it will help answer my next question.\n- The ideas presented in this work are extremely similar to another concurrent work on building a WebVoyager SOTA web agent, Agent-E. There is almost a one-to-one mapping between the proposed changes (both action and observational space) in AgentOccam and Agent-E. \ni. The simplifying of action space in AgentOccam is close to the selection of \"primitive actions\" in Agent-E.\nii. The Planning tree in AgentOccam is close in principle to \"hierarchical planning\" in Agent-E.\niii. Simplifying Web Page observation in AgentOccam is same in principle to the idea behind \"DOM distillation\" in Agent-E.\niv. Selective replaying of observation history in AgentOccam is similar in principle to \"change observation\" in Agent-E.\nAnd both agents are SOTA on different benchmarks, AgentOccam on WebArena and Agent-E on WebVoyager, and hence would be good to have a detailed comparison, both quantitative and qualitative, between them.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "The following are the major strengths of the paper:\n-  They have focused on the first principles of building a domain-specific agent, i.e., a web agent and hence utilized domain-knowledge to optimized how LLMs can be used in the best way for this domain.\n- The ablations presented are very well presented allowing the reader to understand the impact of each of the changes proposed in the paper.", "weaknesses": "- The results are shown on WebArena which is a simulated environment. It would be more realistic to see how it performs on WebVoyager which is more realistic in terms of how web behaves.\n- The tree representations used for planning needs better explanation. I have to honest that I couldn't grasp it as well as I would want to. While I understand it in principle, presenting it with a more detailed example, even if in Appendix, might help someone like me look into the details of it.\n- While the methodology/process presented in this paper of \"optimizing/aligning LLMs to the domains when designing agents\" is very important and valuable, it is worth noting that the specific proposed changes themselves (understandably) don't generalize to other domains beyond web.\n- Minor typo in title of Sec 5 (not a weakness obviously but would be good to fix in the final version).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VdiBvEQ50R", "reviewer": "Reviewer_LC8w", "summary": "The paper proposes a new method for improving LLM-based web agents. The contributions are mainly two folds:\n\n1. Improve alignment between the action and observation space of the web agent and the data on which the LLM has been pre-trained. The authors propose heuristics to: a. Simplify the actions available to the agent and b. Better clean and format the web page observation input for the LLM.\n\n2. New actions and improvement for planning. The authors introduce new `branch` and `prune` actions for altering the plan, as well as prune the web elements included in the historical context.\n\nThe authors conduct experiments with WebArena, a web agent benchmark that includes simulation environments across 5 websites.\n\n- The proposed method obtains significant improvement over the base agent design from the original WebArena.\n- The proposed method also outperforms the SOTA method on WebArena by 6 points. (Note that the base model is not the same)\n- Through ablations, both the action/observation alignment and improvement in planning contribute to the overall bump in performance, with the most gain coming from the action/observation simplification.\n- The proposed method can also be combined with another method though the improvement depends on specific agent design. In Step, the proposed method brings improvement compared with Step, though it does not bring further improvement to AgentOccam. When combined with an LLM judge, the authors observe further improvement over AgentOccam.", "questions": "1. Are there experiments on more datasets to show the effectiveness of the proposed heuristic beyond WebArena?\n\n2. Are there experiments with other LLMs.\n\n3. If possible, it would be better to include results using the same base LLM. It is okay to report SOTA on the best LLM available. But for fair comparison and a better understanding of the contribution of the proposed method, it would be very helpful to include results using the same LLM as the baselines.\n\n4. I recommend swapping Figure 5 and Table 10 for the ablation results. The numbers in the table is easier to consume.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The overall experiment results are positive, and new SOTA performance was obtained on the WebArena benchmark.\n- The proposed design is simple yet effective, so it should be rather easy to be applied.\n- The experiment on WebArena is extensive, with a comparison of multiple base models and ablations to show improvement of individual components.", "weaknesses": "- The main contribution, and also the main source of improvement, is the simplification of action and observation space. A natural question would be how well these simplifications could be generalized across websites and tasks. Although WebArena contains multiple tasks, it still only contains five websites. To further verify the effectiveness of the proposed method, I would suggest including experiments on other datasets, e.g., Mind2Web, which has more websites.\n\n- Similarly, the alignment could potentially be LLM-dependent. The paper only conducted experiments with GPT-4; it would be better if it could be shown that the method could generalize to different LLMs, which potentially have seen different mixes and formats of training data.\n\n- How to properly represent webpage observation and select the action space has been studied by a few works, both for web agents and web IE, in the past (1) (or, in general, all papers on web agents have touched on this topic). It is appreciated that the authors find a new representation that works very well, but the technical novelty is a bit limited, especially for conferences like ICLR.\n\n1. Zhou Y, Sheng Y, Vo N, Edmonds N, Tata S. Simplified DOM Trees for Transferable Attribute Extraction from the Web.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KvSXwykYkP", "reviewer": "Reviewer_Tca3", "summary": "This paper proposes a new pathway to improve the performance of LLM-based web agents. Compared with previous works on web agents, it mainly focuses on optimizing the action space and observation space, rather than designing reasoning/planning/memory modules for more complex inferences. The performance on WebArena shows the effectiveness of AgentOccam on web tasks.", "questions": "Please see the cons. Please point out if I have any misunderstandings, and I'm willing to improve my rating if the author could greatly address my concerns.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "Pros:\n- The perspective to improve the capability of web agents is great, which it dives into the action space and observation space. It is a common pathway that may take effect in not only web tasks but also other tasks with complex actions and observations.\n- I think the action optimization is inspired, especially for abstracting and generating new actions. From my perspective, it is similar to the tool learning problem and can be effective.\n- The improvement in performances seems great, where it demonstrates a huge improvement on WebArena.", "weaknesses": "Cons:\n- The author claims to solve the generalizability across all real-world applications in the abstract. However, the method section seems to just illustrate how to deal with the specific web environment. There are too many 'specifically' and 'in particular' in sections 4.1 and 4.2. Does it mean I need to design different merges manually, where the method is just providing a guide or strategy to conduct the manual process? I think an ideal optimization is to conduct the optimization automatically, even by prompting LLMs. You may provide concrete examples of how your approach could be applied to other domains, or discuss any limitations in generalizability.\n- I think a general method to automatically conduct the alignment is necessary, especially for the action space mentioned above. But this work seems to fail to do so.\n- I'm confused about the part 'Replaying Observation History Selectively', could you please demonstrate with more details? I think this is an information retrieval process, but I'm aware of how to get relevant information. You may provide a step-by-step example of how the selective replay process works on a sample task. This would help clarify the information retrieval aspect that I'm unsure about. Additionally, you might show more details on how the relevance of information is determined and how this process differs from or improves upon standard information retrieval techniques.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)