AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents

## Abstract

Abstract Autonomy via agents based on large language models (LLMs) that can carry out personalized yet standardized tasks presents a significant opportunity to drive human efficiency. There is an emerging need and interest in automating web tasks (e.g., booking a hotel for a given date within a budget). Being a practical use case itself, the web agent also serves as an important proof-of-concept example for various agent grounding scenarios, with its success promising advancements in many future applications. Meanwhile, much prior research focuses on handcrafting their web agent strategies (e.g., agent’s prompting templates, reflective workflow, role-play and multi-agent systems, search or sampling methods, etc.) and the corresponding in-context examples. However, these custom strategies often struggle with generalizability across all potential real-world applications. On the other hand, there has been limited study on the misalignment between a web agent’s observation and action representation, and the data on which the agent’s underlying LLM has been pre-trained. This discrepancy is especially notable when LLMs are primarily trained for language completion rather than tasks involving embodied navigation actions and symbolic web elements. In our study, we enhance an LLM-based web agent by simply refining its observation and action space, aligning these more closely with the LLM’s capabilities. This approach enables our base agent to significantly outperform previous methods on a wide variety of web tasks. Specifically, on WebArena, a benchmark featuring general-purpose web interaction tasks, our agent AgentOccam surpasses the previous state-of-the-art and concurrent work by 9.8 (+29.4%) and 5.9 (+15.8%) absolute points respectively, and boosts the success rate by 26.6 points (+161%) over similar plain web agents with its observation and action space alignment. We achieve this without using in-context examples, new agent roles, online feedback or search strategies. AgentOccam ’s simple design highlights LLMs’ impressive zero-shot performance on web tasks, and underlines the critical role of carefully tuning observation and action spaces for LLM-based agents.

## 1 Introduction

AI agents leveraging large language models (LLMs) show great potential in automating repetitive and programmatic tasks and thereby alleviating human workloads (Gao et al., ; Xi et al., ; Yang et al., ). LLMs showcase remarkable capabilities in perception, reasoning and planning primarily due to their pre-training and post-learning.
However, their effectiveness is significantly constrained when task-specific observation and action representations diverge from the parametric knowledge encoded during training of LLMs. For instance, in web-based tasks, these agents perform notably below human levels (Zhou et al., ; Koh et al., ).

Figure: Figure 1: Overview of AgentOccam. Unlike prior research that works intensively on designing compound LLM policies, we enhance the web agent simply by aligning the web interaction action and observation space with the functioning LLM’s acquired knowledge and skills during its training.
Refer to caption: x1.png

To improve web task performance by LLM-based agents, recent work focuses on designing better agent policies with either handcrafted prompting templates (Sodhi et al., ) or hard-coded auto-prompting strategies (Fu et al., ; Wang et al., ). While those pre-defined strategies can be effective for certain tasks, they struggle to generalize to diverse websites and varying skill requirements. Another emerging trend is to adopt sampling or search algorithms for a dynamic exploration of web navigation actions, which reduces dependence on pre-defined strategies but increases the cost of LLM inferences. (Koh et al., ; Zhang et al., ; Pan et al., ).

In this work, we aim to enhance an LLM-based web agent’s proficiency by optimizing the text-based task understanding and reasoning of existing LLMs, rather than refining the agent strategies. Automating web tasks is challenging, as the agent needs to i) accurately extract information from web pages with varying formats and encoded scripts, and ii) issue appropriate embodied actions, selecting from those defined merely on web (e.g., scrolling, clicking, or hovering over buttons). These web observation and action spaces are less common in both, the pre- and post-training data of LLMs, preventing the LLMs from fully realizing their potential in accomplishing general-purpose web tasks. Therefore, we study how to properly tune the observation and actions for LLM-based web agents, to align them with the functioning LLMs capacities learned during training.

As shown in Figure 1, our method comprises of three components: i) We reduce non-essential actions to minimize the agent’s embodiment and trivial interaction needs; ii) We refine the observation by eliminating redundant and irrelevant web elements, and restructuring web content blocks for more succinct yet as informative representations; iii) We introduce two planning actions (branch and prune), which enables the agent to self-organize navigation workflow with a planning tree, and use the same structure to filter the previous traces for history replay. We implement these components by generic rules that applies to all types of markup-language-formatted web pages, without leveraging task-related information on the test benchmark.

By combining the three techniques mentioned above, our proposed agent AgentOccam performs substantially better on web tasks across websites in the WebArena environments (Zhou et al., ). AgentOccam outperforms the previous state-of-the-art approach by 9.8 absolute points ($+29.4\%$) and surpasses concurrent work by 5.9 absolute points ($+15.8\%$). Notably, unlike most prior work, we do not use any in-context examples, additional online search or sampling, nor specialized prompting templates or agent roles to play well. In contrast, AgentOccam delivers such strong performance with an unexpectedly simple approach: letting the LLM issue actions within the processed and augmented observation and action spaces. Compared with a similar plain web agent without these proposed observation and action space changes, AgentOccam increases the success rate by 26.6 absolute points ($+161\%$).

In summary, the primary contribution of this work are as follows. First, we develop a new state-of-the-art agent, AgentOccam, for web tasks. On the WebArena benchmark consisting of 812 tasks across five diverse websites (e.g., shopping, searching on a forum), AgentOccam outperforms previous and concurrent work significantly.
Second, we shed light on the strong zero-shot performance of LLMs on web tasks with our simple agentic workflow, in sharp contrast to many more complex compound agent policies.
Last, our work on aligning the observation and action spaces is orthogonal to agentic strategies and can be combined with future advances in that aspect.

**Table 1: Comparison of essential components for different LLM-based web agents.**
| Essential Components | Task-specific Strategies | Additional Module | In-context Examples | Offline Data | Online Search |
| --- | --- | --- | --- | --- | --- |
| AutoGuide (Fu et al., ) | no | yes | yes | yes | no |
| SteP (Sodhi et al., ) | yes | yes | yes | no | no |
| AutoRefine (Pan et al., ) | no | yes | yes | yes | yes |
| LM-Tree Search (Koh et al., ) | no | yes | yes | yes | yes |
| AWM (Wang et al., ) | no | yes | yes | yes (^1^1footnotemark: 1) | no |
| WebPilot (Zhang et al., ) | no | yes | yes | no | yes |
| AgentOccam | no | no | no | no | no |

## 2 Related Work

#### LLM-based Web Agent

Advances in large language and multi-modal foundation models have significantly boosted the development of autonomous agents to solve web tasks. Techniques translating LLMs to powerful decision-making agents (Yao et al., ; Shinn et al., ) have led to progress in web agents, and have inspired many techniques that design inference time agent strategies. Many prior approaches improve the agent system by designing modular systems with specialized LLMs or roles, aiming to break down complex tasks (Sun et al., ; Prasad et al., ). Other works leverage LLMs to extract common patterns from examples or past experience (Zheng et al., ; Fu et al., ; Wang et al., ). However, this line of work often relies on pre-defined control hierarchy, prompt templates or examples to act accurately in the test environments. For example, SteP (Sodhi et al., ) utilizes a stack-based approach for dynamic multi-level control in the web tasks but relies on task-specific atomic policies with environment-related information hard-coded in prompt template. Another line of work focuses on improving web agents’ performance by leveraging more online examples from the environments. Many of them (Zhou et al., ; Zhang et al., ; Putta et al., ) adapt Monte Carlo Tree Search (MCTS) methods, expanding intermediate states (tree nodes) in one task repeatedly by multiple trials over that task. Among them, WebPilot (Zhang et al., ) also adds a global optimization layer for high-level planning. Koh et al. () use a trained value function to guide search and to back-trace on the task execution tree. Auto Eval and Refine (Pan et al., ) trains a separate evaluator, and improves the task execution using reflective thinking (Shinn et al., ) on past trials in the same task. However, sampling or resetting multiple times in the same task, not only increases the inference cost significantly, but also limits its applicability when failed task is not revocable. As a comparison, we highlight the simplicity of our method and its difference with related agent approaches in Table 1.

#### Fine-tuned or Trained Models for Web Tasks

Fine-tuning language or multimodal models for web tasks is another effective approach to enhance decision-making capabilities on the web tasks (Yin et al., ; Hong et al., ; Lai et al., ; Putta et al., ). While fine-tuning promises more adaptivity and broader optimization space, the size of task-specific fine-tuned models are typically not comparable with the most powerful closed-source models. As for training models to follow natural language command on the computer or the web, there is also some early research before LLMs emerged, using semantic parsing (Artzi & Zettlemoyer, ), reinforcement learning (Branavan et al., ) and imitation learning (Liu et al., ; Humphreys et al., ). However, those fine-tuned agents, limited by the base model’s capacities or training data volume, often fail to match those constructed with LLMs regarding performance or/and generalizability, and is beyond the scope of this work.

#### Simulated Web Agent Environments

Web agent development has been supported by increasingly complex web simulators for training and evaluation. These range from basic platforms like MiniWoB (Shi et al., ) and its extension MiniWoB++ (Liu et al., ), to more sophisticated environments such as WebShop (Yao et al., ), WebArena (Zhou et al., ), and VisualWebArena (Koh et al., ). These simulators progressively incorporate real-world complexities, from simple form-filling to tasks across multiple full-featured websites. In this work, we focus only on the text modality, and use WebArena to evaluate our method’s task success and generalizability as it contains different types of websites and task-intents in a single suite.

## 3 Problem Formulation

We formalize the web interaction process by a Partially Observable Markov Decision Process (POMDP, Littman (); Spaan ()): $\langle\mathcal{O},\mathcal{S},\mathcal{A},P,R,p_{0},\gamma\rangle$. In POMDPs, an observation $o\in\mathcal{O}$ consist of information that the agent receives from the web environment, e.g. HTMLs, as well as any instructions and prompts. In this work, we only consider the text modality. A state $s\in\mathcal{S}$ denotes the whole underlying (unobserved) state of the agent and the environment such that the state transition is Markovian. An action $a\in\mathcal{A}$ is either a command recognized by the web environment, or any other unrecognized token sequence that will lead to a stay in the current state. $P$ denotes a deterministic state transition function that records the change in the webpage state given the current state and agent action. $R$ is the reward function that decides the success or failure of the agent’s sequence of actions. In the WebArena environment used in our work, the reward is only assigned at the end of an agent-web interaction episode. $p_{0}$ denotes the initial state distribution which is uniform over 812 tasks in WebArena and discounting factor $\gamma$ is set to 1.

To solve POMDP, a common goal is to find a decision policy $\pi(a_{t}|h_{t})$ maximizing the expected cumulative reward, where $h_{t}$ denotes the observation history $\{o_{0},o_{1},...,o_{t}\}$. In LLM-based web agent design, that is translated to designing a policy $\pi(a_{t}|h_{t})$ with the help of one or more base LLM policy $\pi_{\textrm{LLM}}$ and a set of algorithmic modules. In this work, we work on a special class of policies that can be expressed as: $\pi(g(a_{t})|h_{t})=\pi_{\textrm{LLM}}(a_{t}|f(h_{t}))$, where $f$ and $g$ are rule-based functions that process the observation (including action instructions) and actions for the LLM policy. We name it the observation and action space alignment problem. Notice that under such problem setting, all of our changes apply only to the observations and the actions. We emphasize not all agent strategies in previous approaches can be represented in this way. For example: search-based algorithms require a control program on the top to select actions and trigger back-tracing; methods with evaluators, reflective thinking or memory modules also necessitate a managing center to alternate between the main LLM and these helper segments or other role-playing LLMs. In contrast, we aim to answer the following question in our work: Can we build a strong web agent with the base LLM policy $\pi_{\textrm{LLM}}$ by optimizing only the observation and action mapping $f$ and $g$?

## 4 Method

Rather than introducing any new modules or hierarchical structures on top of the base LLM, our method focuses on a simple web agent workflow that inputs the web observations to a general-purpose LLM-API and uses the LLM outputs as actions directly.
In this section, we describe the process of aligning web tasks, which necessitates embodiment knowledge, with the predominantly static and text-centric nature of LLM training.
Section 4.1 discusses our strategies (summarized in Figure 2) for refining the action space to be more compact and reducing the need for the agent’s embodiment capabilities. Section 4.2 outlines our methods (summarized in Figure 4) for condensing web content descriptions to be both brief and informative, and identifying key web elements and relevant steps for retention to organize the agent’s memory in a pertinent manner.

### 4.1 Action Space Alignment

Figure: Figure 2: In aligning the action space with LLM pre-training, we only retain high-utility actions and lessen the demand for advanced embodiment skills (steps 1 and 2). Additionally, we incorporate planning steps, allowing the agent to autonomously manage task breakdown and execution (step 3).
Refer to caption: x2.png

A web agent’s action space defines the valid commands it can use to interact with web environment. The WebArena simulator supports translating three categories of actions into mouse and keyboard operations: basic actions (e.g., click, type), tab operations (e.g., tab_focus for managing active tabs), and page operations (e.g., go_back for navigation). These actions are detailed in Appendix A, along with a comparison of our changes to the action space.

Based on our observation of common failure modes in web agents, there are two key problems that need to be solved by editing the action space: i) removing irrelevant actions that LLMs struggle to understand and frequently misuse, and ii) improve the memorization and planning ability when the task execution requires navigating multiple potential paths to successfully execute.
We propose that the first can be corrected simply by removing and combining actions. The second issue was often addressed in the previous work using handcrafted rules or strategies, making these approaches hard to generalize. In this work, we address the second problem by introducing actions that allow the LLM to autonomously generate plans and manage the task workflow. These proposed solutions are explained in details below and in Figure 2.

Simplifying the Action Space.
First, we eliminate actions that can be replicated using similar actions or replace multiple actions by one action with the same expressiveness (illustrated in Figure 2 step 1). Specifically, we remove the noop action, signifying “no operation”, as a distraction to the agent in most cases. Similarly, tab operations, which manage the focusing, opening, or closing of tabs are removed because they are only needed in a limited cases of multi-site tasks requiring two tabs. Furthermore, we limit page navigation actions like go_forward and goto, as their utility is greatly constrained by the agent’s poor memory of the relationship between a page’s URL and its content. By eliminating these less effective actions, our goal is to minimize distractions and boost the agent’s concentration on more meaningful operations. In addition, we introduce the note action, allowing the agent to record key observations for subsequent conclusions, and the stop action, enabling the agent to autonomously conclude the trajectory with answers. We also add a go_home command for multi-site tasks, enabling the agent to navigate directly to the homepage where all available sites are listed.

Second, we eliminate actions that heavily require embodiment knowledge and simplify low-level actions into more abstract operations as shown in Figure 2 step 2. In particular, we reduce commands that LLM-based agents struggle with unless provided with detailed context-specific guidance, like hover or press (the latter is for pressing key combinations, often shortcuts).
To properly use these actions requires LLMs to have embodied thinking of the current scenario, especially regarding the mouse position or keyboard operations, which it has not acquired during the training. Additionally, we remove the scroll action, opting instead to load the full page content as the web state. This change is in response to our observation that agents tend to engage in aimless and repetitive scrolling when an essential link is not visible at the top of the page, wasting steps without making progress. Furthermore, we streamline the agent’s interaction with drop-down menus; instead of selecting the menu and then an option, a single click command with the ID of desired option now suffices. The list of all actions in original and reduced action space are shown in Table 3, together with the frequency they are taken in different agents.

Planning via Generation.
Web tasks often requires solution that requires navigating multiple paths (e.g. extracting information from one page and submitting it to another page, like the task of creating a refund request on the contact us page for a broken product (task template 154), which requires parsing the order ID and refund amount from the order pages).
We introduce two actions (branch and prune) to generate plans in a tree structure and save them for future observations. As Figure 2 step 3 shows, the LLM-generated plans starts with a root node being the objective of the task. The branch action will generate new subplans under the current node, decomposing high-level objectives into smaller, more manageable subgoals. Conversely, the prune action allows the agent to give up the current sub-plan (often after repetitive failed attempts) and seek for alternatives. Together with the branch and the prune actions, the LLM can edit the planning tree autonomously.
Note that these two planning actions are of no difference from the native navigation actions in the web environment (e.g. click, type) and the LLM is free to choose when to take these actions to update the plan. The generated plan provides a context for future action generation and enhances the consistency of actions in one trajectory.
This approach leverages the intrinsic planning ability in LLM itself. We argue that this will not compromise the agent’s generalization capability as this design relies minimally on prior knowledge.

### 4.2 Observation Space Alignment

Figure: Figure 3: The components of our web navigation agent’s prompt. It includes a general instruction outlining the task, the desired output and available actions, as well as online task information providing the current goal, the agent’s past interactions, and the latest observations. Notably, the sections on previous interactions and current observation use the most tokens. These can be attributed to two main factors: the length of the pages and the extent of history span, with current observation primarily depending on page length and past interactions on both page length and history span.
Refer to caption: x3.png

Figure: Figure 4: To align the task’s observation space with the base model’s pre-training, we condense a single-page length by removing unnecessary texts that repetitively describe the web page’s functionality and layout (step 1), and by identifying page elements relevant to the task for the agent to remember (step 2). Additionally, we optimize the agent workflow memory through a planning tree, viewing each new plan as a separate goal and excluding past steps’ information dedicated to previous plans to enhance memory conciseness (step 3).
Refer to caption: x4.png

The observation space of web agents consists of task objectives, instructions, previous interaction, the current web text descriptions or screenshots (see Figure 3 and Appendix D for our agent). Among them, previous interactions and current web content consumes the most number of tokens, which scales with the length of a single page and the length of history. This often results in a long context window, which not only increases the LLM inference cost but also poses challenges for LLM to extract related information accurately. Therefore, our main goal in refining the observation is to address these two aspects. Additionally, the alignment of observations is outlined in Figure 4.

Simplifying Web Page Observations.
The content on web pages are represented in HTML or accessibility tree format in most text-only web agents. These formats are designed towards front-end loading and rendering, containing numerous formatting tokens making them lengthy and repetitive, as illustrated in Figure 4 Step 1. Our goal is to optimize the representation to make it more readable to LLMs in one single page. Specifically, we merge function-descriptive web elements (e.g., StaticText [761] ‘My Account’) with interactive elements that share the same label (e.g., link [1312] ‘My Account’). We then convert table and list blocks to Markdown, eliminating repetitive structural tokens (e.g., columnheader, gridcell). Consequently, we achieve a more concise representation while keeping the same information.

Replaying Observation History Selectively.
Taking observation history as input is important for decision-making agents to act consistently for tasks requiring long horizons, given that the observation state only contains partial information about the environment’s state. For web tasks, it is also important to include both observation and action history as some key information may not be displayed on the current page. However, the observation history will also significantly scale up the context length and increase reasoning difficulty as well as inference cost. We address this issue by only selecting the most important and related information on previous web page, according to two rules based on the “pivotal” nodes (defined later) and the planning tree.

First, we observe that only small amount of content on a web page is pertinent to a specific task among several steps and is worth to replay in future steps. For example, in tasks requiring the agent to find all reviews within three months, it is unnecessary to keep other reviews or some unrelated links like Contact Us on the page. Thus we employ a simple rule to identify this small amount of content by leveraging the tree structure of web data (e.g. accessibility tree). To do this, we first instruct the agent to pinpoint the crucial web elements denoted as “pivotal” nodes, at the same time when the agent generates an action. The agent is then programmed to include only the pivotal nodes’ ancestor nodes (indicating their global hierarchy and position), sibling nodes (providing immediate context), and descendant nodes (offering detailed characteristics) in the future observations as illustrated in Figure 4 Step 2. This effectively narrows down the volume of data and level of noise passed to future context of LLM inference.

Second, we observe that not all previous steps’ observation needs to be noted during the inference of future step. Thus we can leverage the planning tree generated by the agent itself to keep the agent’s focus sharp. Specifically, when the agent initiates a branch action to develop a new plan, we treat this new plan as a separate goal. Steps taken for earlier plans and their observations will be dismissed in the current plan’s observation window, as depicted in Figure 4 step 3. This allows the agent to focus only on information dedicated to the current plan for a sub-task.

## 5 Experimental resuts and analysis

#### Environment.

We utilize WebArena (Zhou et al., ), an interactive web simulator, as our benchmark. WebArena consists of fully functional websites from four common domains: e-commerce platforms (OneStopShop), social forums for idea and opinion exchange (Reddit), collaborative software development (GitLab), and content management for creation and management of online data (online store management).
The platform additionally includes utility tools: a map, a calculator and a scratchpad, and Wikipedia to enable human-like task-solving. The benchmark consists of 812 tasks generated from 241 templates. A template here is a parametric form of a task intent, allowing for multiple instantiations with different keywords. Each task is accompanied by a evaluator/reward function that programmatically checks the correctness of the final information with respect to the desired ground truth information(^2^22We identified and corrected errors in the original evaluators, with details discussed in Appendix B. Our approach outperforms the baseline methods with both original or corrected evaluators.). We use GPT-4-turbo-2024-04-09 (Achiam et al., ) to build our AgentOccam.

#### Baselines.

We compare AgentOccam with the following prior and concurrent work: 1) WebArena agent: the Chain-of-Thought (CoT) prompted agent included in the WebArena benchmark (Zhou et al., ). 2) SteP (Sodhi et al., ): a stack-based approach on top of 14 human-written atomic strategies tailored to solving WebArena. 3) WebPilot (Zhang et al., ): a multi-agent, multi-level MCTS based agent that reports state-of-the-art overall performance on WebArena. 4) Agent Workflow Memory (AWM) (Wang et al., ): a method automatically summarizing workflow from past experience. SteP has made their code and interaction trajectories public. Hence, we are able to fully replicate the agents from WebArena and SteP with GPT-4-turbo in the identical web environments as our methods, for a fair comparison.(^3^33In our experiments, we note that all agents occasionally fails due to errors from the WebArena simulator, such as posting rate limits in Reddit or login expiration. In such cases, we restart the experiments.) WebPilot and AWM, being concurrent works with this paper, have not yet provided source code or resulting trajectories, limiting our analysis of these works to just reporting the aggregated performance numbers included in their technical reports. Our analysis focuses on SteP as it is the most performant method prior to this work.

**Table 2: Comparison of the success rate (SR) of AgentOccam with baseline agents on WebArena.**
| Agent | Model | SR (%) | Shopping | Shopping Admin | GitLab | Map | Reddit | Multisite |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  |  | (#812) | (#187) | (#182) | (#180) | (#109) | (#106) | (#48) |
| WebArena-replication | GPT-4-Turbo | 16.5 | 16.6 | 15.9 | 10.0 | 22.9 | 21.7 | 16.7 |
| SteP-replication | GPT-4-Turbo | 33.3 | 33.2 | 32.4 | 26.7 | 35.8 | 52.8 | 12.5 |
| AWM | GPT-4 | 35.5 | - | - | - | - | - | - |
| WebPilot | GPT-4o | 37.2 | - | - | - | - | - | - |
| AgentOccam | GPT-4-Turbo | 43.1 | 40.6 | 45.6 | 37.8 | 46.8 | 61.3 | 14.6 |

#### Question 1: How well does AgentOccam perform?

As seen from the results in Table 2, our agent AgentOccam, which optimizes the action and observation space, now sets a new SOTA on the WebArena benchmark. It increases the overall success rate from 37.2% to 43.1%, a 15.8% relative improvement over best results among previous and concurrent work. We observe that AgentOccam not only accomplishes tasks in the template that is previously unsolvable, like updating personal information on OneStopShop (task template 165), but it also raises the success rate for templates with mixed results previously, such as setting a homepage URL on a GitLab profile (task template 331). This is further illustrated in Figure 6 in the appendix.

#### Question 2: How much does each observation and action space change contribute to AgentOccam ?

Figure: Figure 5: Ablation study of AgentOccam’s action and observation space refinement. We incrementally add refinement components and evaluate their marginal performance gains.
Refer to caption: x5.png

**Table 3: Action statistics for the ablation study of AgentOccam’s components. Each number in the table represents the frequency of an action across all the tasks within the experiment setting. Actions noop, go_forward, tab_focus and tab_close are not included since they are not used even once in vanilla agent and removed in our method.**
| Exp. | click | hover | type | press | scroll | new_tab | go_back | goto | note | stop | go_home | branch | prune |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Vanilla | 2328 | 126 | 1024 | 7 | 132 | 20 | 71 | 511 | - | -(^4^4footnotemark: 4) | - | - | - |
| $\downarrow$ Actions | 7119 | - | 2531 | - | 370 | - | 52 | - | 194 | 512 | 36 | - | - |
| Above + X Scrolling | 7033 | - | 2390 | - | - | - | 100 | - | 219 | 536 | 42 | - | - |
| Above + Obs Opt. | 6890 | - | 2040 | - | - | - | 56 | - | 201 | 571 | 23 | - | - |
| Above + History | 4625 | - | 1286 | - | - | - | 94 | - | 112 | 801 | 54 | - | - |
| AgentOccam | 4720 | - | 1159 | - | - | - | 339 | - | 197 | 769 | 42 | 34 | 47 |

**Table 4: Average observation tokens per step across WebArena sites. We use the gpt2 tokenizer from Huggingface (Radford et al., ).**
| Exp. | All | Shopping | Shopping Admin | GitLab | Map | Reddit | Multisite |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Vanilla | 2210.2 | 2272.1 | 2460.2 | 2199.1 | 1883.2 | 2132.4 | 1751.0 |
| $\downarrow$ Actions | 1652.0 | 1644.7 | 2133.1 | 1981.3 | 912.0 | 1081.2 | 1296.8 |
| Above + X Scrolling | 3376.2 | 3148.0 | 5403.7 | 3364.9 | 1378.1 | 2603.6 | 1975.5 |
| Above + Obs Opt. | 2891.1 | 1722.5 | 4791.7 | 2560.8 | 1476.4 | 3332.3 | 1619.4 |
| Above + History | 3051.3 | 1802.6 | 5140.2 | 3153.3 | 862.1 | 3156.1 | 2030.3 |
| AgentOccam | 2930.9 | 1634.2 | 4920.7 | 3126.8 | 1056.0 | 3697.8 | 1282.5 |

**Table 5: Average number of steps per task across all WebArena sites.**
| Exp. | All | Shopping | Shopping Admin | GitLab | Map | Reddit | Multisite |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Vanilla | 6.2 | 6.2 | 6.6 | 5.9 | 5.7 | 7.4 | 4.4 |
| $\downarrow$ Actions | 13.3 | 10.6 | 14.3 | 14.8 | 11.9 | 15.2 | 13.7 |
| Above + X Scrolling | 12.7 | 9.0 | 14.0 | 14.8 | 12.7 | 13.0 | 14.0 |
| Above + Obs Opt. | 12.0 | 8.5 | 13.2 | 15.4 | 10.2 | 12.1 | 13.2 |
| Above + History | 8.6 | 5.6 | 9.6 | 10.3 | 8.3 | 7.6 | 12.9 |
| AgentOccam | 9.0 | 6.7 | 9.2 | 10.8 | 8.5 | 8.6 | 13.4 |

We evaluate the contribution of each component in AgentOccam described in Section 4 to its overall success by incrementally integrating them into the vanilla agent (WebArena-Replication) and assessing the marginal performance gain shown in Figure 5. The details of each incremental experiment are as follows:

i) Removal of non-essential actions ($\downarrow$ Actions): Narrowing the action space can reduce the level of distraction for LLM policies and significantly improves performance across all tested websites as shown in Figure 5. By removing rarely used actions like tab_focus,go_forward, hover and press, the agent spends less steps wandering around and explores more efficiently using actions such as click and type. Table 3 shows it reduces hundreds of hover and goto actions while significantly increase the number of click and type.

ii) Disabling scrolling (Above + X Scrolling): We observe that LLM policies tend to use scroll up and down often when they do not know what to do (since these action are revertible).
Consequently, it significantly delays the task execution and causes looping over in certain tasks.
As a result, disabling the scrolling action and passing the entire page to agent proves advantageous, especially for GitLab and Reddit tasks. However, this strategy increases the number of observation tokens, which will be addressed by subsequent refinements.

iii) Simplifying web page elements (Above + Obs Opt.): We remove redundant text and web format as show in Figure 4 Step 1. This results in
fewer tokens in the context window, as outlined in Table 4. It helps the agent focus on web elements crucial to task success across all websites and boosts the performance on all task types, except on Gitlab, where this sometimes leads the agent to overlook simpler solutions (task id 394).

iv) Selective replay of web elements in one page (Above + History): In this experiment, we follow step 2 shown in Figure 4 to add a subset of elements from previous web pages as history. We observe that it allows the agent to avoid repetitive actions in tasks, significantly decreasing the steps needed for task completion as demonstrated in Table 5. However, this addition slightly hurts performance in tasks with dense single-page content or those requiring navigation across multiple pages, as shopping and Reddit tasks success rate drops by 3.2 and 6.0 points.

v) Planning via generation and selective replay of past pages (AgentOccam; Above + Planning): We introduce actions branch and prune to generate actions and exclude historical steps not in the current sub-plan from the current prompt context. This results in performance gains in tasks across nearly all websites, alongside a reduction in the required observation tokens. The actions branch and prune are both primarily used in correcting a failed strategy and trying an alternative path. For example, in the task of identifying the nearest national park to Boston (task id 265), the agent employs a branch action to adopt an alternative search strategy after a failed search attempt. In a GitLab task (task id 563), after multiple failed attempts using the Create project button, the agent opts for a prune action to explore other methods.

#### Question 3: Could the power of AgentOccam be combined with other agentic strategies?

**Table 6: Success rate (SR) of AgentOccam combined with agent strategies on WebArena.**
| Agent | Model | SR (%) | Shopping | Shopping Admin | GitLab | Map | Reddit | Multisite |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  |  | (#812) | (#187) | (#182) | (#180) | (#109) | (#106) | (#48) |
| AgentOccam | GPT-4-Turbo | 43.1 | 40.6 | 45.6 | 37.8 | 46.8 | 61.3 | 14.6 |
| AgentOccam + SteP | GPT-4-Turbo | 41.1 | 46.5 | 36.3 | 36.7 | 47.7 | 50.9 | 18.8 |
| AgentOccam + Judge | GPT-4-Turbo | 45.7 | 43.3 | 46.2 | 38.9 | 52.3 | 67.0 | 16.7 |

A natural question to ask next is if we can combine these changes with other common agent strategies or prior work, since the changes in observation and action space are orthogonal and complementary to them.
We showcase two example studies to answer this question: one with the SteP method (Sodhi et al., ) and another action selection method with LLM-as-a-judge.

The judge method is motivated by our observation of the high variation from the agent’s behavior. In some key steps, the agent has certain probability of generating the correct action but often failing to do so, making it hard for the agent to recover from later pages.
For instance, when tasked with identifying the most suitable subreddit for posting (task template 6100), the AgentOccam agent tends to hastily choose less relevant subreddits and gets stuck there. To address this, we direct the AgentOccam to generate all possible suitable actions instead of one action at each step. These action candidates are then evaluated by another LLM (GPT-4-turbo as well) prompted to be play the role of a judge and select the best action. The prompts for the judge are included in Appendix D.

Table 6 shows that a AgentOccam + SteP agent, enhanced with task strategies, outperforms the standalone SteP method but doesn’t match AgentOccam’s base performance. Additionally, combining AgentOccam with a judge role through an action prediction and selection pipeline rectifies some of the base agent’s behavioral misconduct.

By analyzing the trajectories of each method, we observe that the task-specific strategy like SteP can help when the strategy fits the task requirement. For example, in the task of "Draft an email to the shop owner via their contact us function for a coupon as {reason}" (task template 163), the AgentOccam + SteP and SteP agents excel by prompting the agent explicitly not to click the submit button after drafting, where AgentOccam fails to follow. However, for tasks outside the designed strategies, these hints can mislead the agent, leading to 2 points drop in overall success rate of AgentOccam + SteP compared to AgentOccam only. An example is task 639, where the agent, guided by SteP’s instruction "Under forums, you will see only a subset of subreddits. To get the full list of subreddits, you need to navigate to the Alphabetical option.", repetitively navigates away from the appropriate subreddit, and generates reasons for its action selection that "Clicking on the ’Alphabetical’ link will help us access a more comprehensive Reddit list.", demonstrating how hard-coded strategies can distract the agent and hurt generalizability.

The AgentOccam + Judge agent, combining the AgentOccam’s generated action list with the second opinion from a LLM judge increases its overall success rate by 2.6%, by completing tasks where it may well fail due to intermediate decision flaws. For example, in choosing the right subreddit for a post (task template 6100), the base AgentOccam might hastily pick from an initial list, whereas the AgentOccam + Judge agent conducts a thorough search using post keywords or explores the entire forum list before drafting the post. This approach minimizes errors due to rushed decisions, increasing the likelihood of successfully completing task series.

## 6 Conclusion

In this paper, we proposed a simple but efficient LLM-based web agent AgentOccam that refines its action and observation spaces to be more comprehensible for LLMs primarily trained on static text. Unlike other methods, AgentOccam stands out for its remarkably simple policy workflow, requiring no extra modules, LLM calls, or in-context examples. This simplicity does not compromise its performance; AgentOccam surpasses previous and contemporary approaches on WebArena by 9.8 (SteP) and 5.9 (WebPilot) absolute points, respectively. Our result emphasize the importance of maintaining a simple agent architecture for better generalizability, echoing the Occam’s razor principle. In summary, AgentOccam aims to lay a solid foundation and offer valuable insights for future web agent research and development.

## Appendix A Comparison of the Vanilla and the Aligned Action Space

**Table 7: The action space in WebArena.**
| Category | Action Type | Description |
| --- | --- | --- |
| Basic Actions | noop | Do nothing |
| click(elem) | Click at an element |  |
| hover(elem) | Hover on an element |  |
| type(elem, text) | Type to an element |  |
| press(key_comb) | Press a key combination |  |
| scroll(dir) | Scroll up and down |  |
| Tab Operations | tab_focus(index) | Focus on the i-th tab |
| new_tab | Open a new tab |  |
| tab_close | Close current tab |  |
| Page Operations | go_back | Visit the last URL |
| go_forward | Undo go_back |  |
| goto(URL) | Go to URL |  |
|  |  |  |

**Table 8: The aligned action space of AgentOccam.**
| Category | Action Type | Description |
| --- | --- | --- |
| Basic Actions | noop | Do nothing |
| click [id] | Click at an element |  |
| hover | Hover on an element |  |
| type [id] [content] | Type to an element |  |
| press | Press a key combination |  |
| scroll | Scroll up and down |  |
| Tab Operations | tab_focus | Focus on the i-th tab |
| new_tab | Open a new tab |  |
| tab_close | Close current tab |  |
| Page Operations | go_back | Visit the last URL |
| go_forward | Undo go_back |  |
| goto | Go to URL |  |
| go_home(^5^55Valid only in multisite tasks.) | Go to home page |  |
| Workflow Management | note [content] | Take notes |
| stop [answer] | Stop with an answer |  |
| Planning Actions | branch [id] [intent] | Generate a new plan |
| prune [id] [reason] | Restore to a previous plan |  |
|  |  |  |

We list the action space of WebArena and our aligned action space in Table 7 and 8, respectively. In detail, we remove non-essential and embodiment-understanding-required actions like noop and scroll, and add more actions for internal workflow management or autonomous planning control.

## Appendix B Evaluator Rectifications

### B.1 Rectification Categorization

We only modify the evaluator when it’s deemed erroneous due to the wrong task labels or misuse of evaluating functions. When the task definition and corresponding evaluation metric match to some extent but might be misleading to most agents and even to human, we still keep the original ones to ensure the slightest reasonable changes. We emphasize that we re-implement WebArena’s base agent SteP’s agent with the same web environment and modified evaluators as AgentOccam for a fair comparison. For example, we keep the evaluators of shopping tasks defined with template 163, requiring the agent to "Draft an email to the shop owner via their contact us function for a coupon as {reason}", which doesn’t explicitly specify whether to submit the drafted email. However, the evaluator is defined to assess the not yet submitted email. All capable LLM-based agents we have tested, which have been aligned to be helpful, will for sure submit the email if not directly prompted to behave in the way the evaluator desires, leading the email field to be blank and thus failing those tasks. Another example of this kind is the Reddit task asking the agent to find the most appropriate subreddit to post (task template 6100), where the assessment of appropriation is very subjective. In all those tasks, we follow the original evaluators, though their evaluation outcomes are arguably questionable.

We categorize our evaluator modifications into two classes, namely label errors and improper evaluation function selection, raise representative examples for each class, and list all the changes made.(^6^66As the evaluator is programmed by the WebArena simulator to be revoked only once at the end of each trajectory, our statement of “our approach outperforms the baseline methods with the original evaluators” refers to setting all the rewards of the trajectories with modified evaluators to be 0, which can be verified with the reported trajectory logs.)

Label errors: We find there exist evaluator definition errors and some typos in the correct answers. In the later cases, the tasks always require exact matching, where any well-aligned LLM-agent would correct those typos in their generation. We thus rectify those errors:

i) Evaluator definitions contain errors. For example, in the Reddit task 584, the evaluator would open up the wrong page for the evaluation. Another case in point is the shopping task 261, where the url_match evaluator is constrained to identifying one correct url (<server_host>:7770/electronics/headphones.html), misjudging the same page (of the identical content) with a different url (<server_host>:7770/electronics.html?cat=60). Tasks fall in this category include: 261-264, 351-355, 584, 644, 679, 707-709.

ii) The answer contains typos or grammatical errors. For example the is car necessary in NYC in task 601, or the budge in task 603. More tasks of this kind include: task id 240, 298-302, 489, 583, 601, 603, 606, 608, 629, 645-649.

Improper evaluation function selection: Evaluator problems are more obvious in this case with the following types:

i) Use the exact_match function that compares whether the answer given by a human label-er and the answer returned by the agent is identically the same. Errors occur when the agent returns a full-form or a more complete answer, where the evaluators’ labels cannot match. For example, in Reddit task 644 that requires the agent to post a meeting notice with the meeting date, where the keyword match for such date is exactly the Dec 15th, where the evaluator would judge other answers like December 15th as incorrect, where we change the keyword matching to one that could match both Dec 15th and December 15th. (In other cases with a single answer, we simply replace exact_match with fuzzy_match, which for instance in task 254 it could match 4125785000 with the agent’s answer The phone number is 4125785000; or replace exact_match with must_include, which for instance in task 363 it could match 778m with the agent’s answer 778 m.) It also demands that the answer should strictly include expressions like virtual meetup, where the agent might add other words in the virtual and meetup. In that sense, we also split the keyword virtual meetup into two separate keywords, i.e., virtual and meetup. Tasks of this kind include: task id 97, 146, 178-182, 254, 293-297, 308-312, 330, 363-367, 415-418, 528-532, 640-649, 653-657.

ii) Use the poorly defined fuzzy_match function, that would view the answer returned as unqualified for the missing-from-expression answer exploration process, or assess answers with more detailed answers as partially correct (reward=0). We thus shift our prompt for the fuzzy_match function from: ‘Help a teacher to grade the answer of a student given a question. Keep in mind that the student may use different phrasing or wording to answer the question. The goal is to evaluate whether the answer is semantically equivalent to the reference answer.” to ‘Help a teacher to grade the answer of a student given a question. Keep in mind that the student has executed the actions to get the answer. They are allowed to use different phrasing or wording to answer the question. The goal is to evaluate whether the key points in the reference answer are included in the student’s answer. We allow answers with additional information that doesn’t contradict the reference answer and review them as fully (not partially) correct.”

iii) Misuse the fuzzy_match function by splitting the keyword list for matching into a list, where each of the keyword and the entire answer, would be evaluated as partially correct (reward=0). In other words, no answer would be assessed as the correct answer (even the gloden-standard answer itself) due to such evaluator function misuse. This could be inferred from the function and the evaluator’s definition. Tasks of this type include: task id 16-20, In such tasks, we simply merge the list of keywords into a string, concatenated with "; ". For instance, for task 16, the previous fuzzy_match field is ["driving: 2min", "walking: 16min"], and we modify it to ["driving: 2min; walking: 16min"].

### B.2 Details

**Table 9: Action statistics.**
| Exp. | click | hover | type | scroll | go_back | goto | note | stop | go_home | branch | prune |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| AgentOccam | 4715 | - | 1159 | - | 339 | - | 197 | 769 | 42 | 34 | 47 |
| AgentOccam + SteP | 5235 | 198 | 1407 | 11 | 25 | 132 | 124 | 1733 | - | - | - |
| AgentOccam + Judge | 4893 | - | 1297 | - | 261 | - | 127 | 726 | 94 | 220 | 41 |

**Table 10: Average number of steps per task across all WebArena sites.**
| Exp. | All | Shopping | Shopping Admin | GitLab | Map | Reddit | Multisite |
| --- | --- | --- | --- | --- | --- | --- | --- |
| AgentOccam | 9.0 | 6.7 | 9.2 | 10.8 | 8.5 | 8.6 | 13.3 |
| AgentOccam + SteP | 11.6 | 10.3 | 12.0 | 10.6 | 12.0 | 14.6 | 11.0 |
| AgentOccam + Judge | 9.4 | 6.7 | 10.5 | 10.6 | 9.6 | 8.4 | 13.5 |

**Table 11: Average observation tokens per step across WebArena sites.**
| Exp. | All | Shopping | Shopping Admin | GitLab | Map | Reddit | Multisite |
| --- | --- | --- | --- | --- | --- | --- | --- |
| AgentOccam | 2932.1 | 1634.2 | 4920.7 | 3126.8 | 1056.0 | 3697.8 | 1282.9 |
| AgentOccam + SteP | 2601.1 | 1675.2 | 3833.3 | 2983.8 | 1196.4 | 3071.4 | 1581.9 |
| AgentOccam + Judge | 2646.4 | 1773.8 | 4181.2 | 2848.4 | 729.7 | 3285.4 | 1433.2 |

## Appendix C Additional Experiment Details

We include the trial statistics for experiments that combine AgentOccam with other compound agent policies like SteP’s strategies and our newly proposed Judge role. Specifically, 9 shows these well performing agent are equally open to web environment exploration, actively issuing environment-changing actions like click and type. Not surprisingly, the AgentOccam + SteP agent frequently issuing un-interactive actions like hover. From Table 10, we can observe that AgentOccam finish the task with the fewest steps, often yielding a task result with 9 steps. Last, from Table 11, those three agents’ token consumptions are of comparative orders of magnitude.

Figure: (a) Shopping.
Refer to caption: extracted/5934875/Figures/shopping_3.png

**Table 12: The success rate (SR) of AgentOccam’s ablation study on WebArena.**
| Agent | Model | SR (%) | Shopping | Shopping Admin | GitLab | Map | Reddit | Multisite |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  |  | (#812) | (#187) | (#182) | (#180) | (#109) | (#106) | (#48) |
| Vanilla | GPT-4-Turbo | 16.5 | 16.6 | 15.9 | 10.0 | 22.9 | 21.7 | 16.7 |
| $\downarrow$ Actions | GPT-4-Turbo | 25.9 | 23.5 | 23.6 | 24.4 | 34.9 | 33.0 | 12.5 |
| Above + X Scrolling | GPT-4-Turbo | 31.7 | 26.2 | 25.3 | 35.0 | 33.0 | 52.8 | 14.6 |
| Above + Obs Opt. | GPT-4-Turbo | 37.1 | 35.8 | 37.4 | 26.7 | 45.0 | 57.5 | 16.7 |
| Above + History | GPT-4-Turbo | 38.2 | 33.7 | 40.1 | 31.7 | 50.5 | 51.9 | 14.6 |
| AgentOccam | GPT-4-Turbo | 43.1 | 40.6 | 45.6 | 37.8 | 46.8 | 61.3 | 14.6 |

As shown in Figure 6, agents that combing AgentOccam with compound agent policies possess different behavioral success patterns. For AgentOccam + SteP, it benefits in tasks where the agent could easily be guided with detailed instructions, such as shopping tasks, with more success (green) blocks and denser success rate in tasks defined with the identical templates. However, it falls short in tasks that require generalizable skills like shopping admin tasks, and in tasks where task-specific strategies distract, like reddit tasks. On the contrary, AgentOccam + Judge agent shares similar patterns with the AgentOccam agent except that some of the success blocks are denser, thanks to the behavior rectification enabled by the action generation and selection pipeline.

In addition, we add the success rate figures of the ablation studies in Table 12, which has been visually represented in Figure 5. During development, we slightly modifies AgentOccam’s prompt such as improving the wording or correcting the typos of the prompts, which don’t affect the semantic meanings of the prompts or add any additional information, and are reflected in the reported trajectory logs. As some failed trajectories are induced by the invalid interaction, we improve the interaction scripts, though not perfectly as it would be beyond the scope of this paper, with the following code shifts:

## Appendix D Agent Prompts

### D.1 AgentOccam

The general prompt template:

- •
With planning

- •
Without planning

Output specifications:

Action space specifications:

- •
Planning action specifications

- •
Navigation action specifications

Observation space example:

### D.2 Judge Used in AgentOccam + Judge Experiments

The general prompt template:

Output specifications:

## Peer Reviews

confidence: 4
contribution: 4
initial_rating: 8
presentation: 3
questions: - I would be curious to see how the agent performs on other benchmarks, especially WebVoyager. This is especially valuable benchmark as besides being more realistic, it will help answer my next question.
- The ideas presented in this work are extremely similar to another concurrent work on building a WebVoyager SOTA web agent, Agent-E. There is almost a one-to-one mapping between the proposed changes (both action and observational space) in AgentOccam and Agent-E. 
i. The simplifying of action space in AgentOccam is close to the selection of "primitive actions" in Agent-E.
ii. The Planning tree in AgentOccam is close in principle to "hierarchical planning" in Agent-E.
iii. Simplifying Web Page observation in AgentOccam is same in principle to the idea behind "DOM distillation" in Agent-E.
iv. Selective replaying of observation history in AgentOccam is similar in principle to "change observation" in Agent-E.
And both agents are SOTA on different benchmarks, AgentOccam on WebArena and Agent-E on WebVoyager, and hence would be good to have a detailed comparison, both quantitative and qualitative, between them.
soundness: 3
strengths: The following are the major strengths of the paper:
-  They have focused on the first principles of building a domain-specific agent, i.e., a web agent and hence utilized domain-knowledge to optimized how LLMs can be used in the best way for this domain.
- The ablations presented are very well presented allowing the reader to understand the impact of each of the changes proposed in the paper.
summary: This work focuses on developing a simple generalized framework for LLM-based web agents, allowing them to leverage the strengths of LLMs to get complete web tasks. This framework focuses on aligning LLMs to the web domain and hence converts the problem to that of finding alignments/mappings in observation and action space between web domain and LLMs. As part of that, for action space alignment, they reduce actions to simple set of actions and add few additional "planning" related actions to the set. For observation space alignment, they propose to simplify the web page representation and also focus on selective representation of history for tracking. They show impressive results of the impact of these strategies on WebArena benchmark while also providing ablations to understand the impact of each of the recommended change. Finally, they show how this simple baseline agent can be promisingly combined with existing strategies.
weaknesses: - The results are shown on WebArena which is a simulated environment. It would be more realistic to see how it performs on WebVoyager which is more realistic in terms of how web behaves.
- The tree representations used for planning needs better explanation. I have to honest that I couldn't grasp it as well as I would want to. While I understand it in principle, presenting it with a more detailed example, even if in Appendix, might help someone like me look into the details of it.
- While the methodology/process presented in this paper of "optimizing/aligning LLMs to the domains when designing agents" is very important and valuable, it is worth noting that the specific proposed changes themselves (understandably) don't generalize to other domains beyond web.
- Minor typo in title of Sec 5 (not a weakness obviously but would be good to fix in the final version).

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: 1. Are there experiments on more datasets to show the effectiveness of the proposed heuristic beyond WebArena?

2. Are there experiments with other LLMs.

3. If possible, it would be better to include results using the same base LLM. It is okay to report SOTA on the best LLM available. But for fair comparison and a better understanding of the contribution of the proposed method, it would be very helpful to include results using the same LLM as the baselines.

4. I recommend swapping Figure 5 and Table 10 for the ablation results. The numbers in the table is easier to consume.
soundness: 3
strengths: - The overall experiment results are positive, and new SOTA performance was obtained on the WebArena benchmark.
- The proposed design is simple yet effective, so it should be rather easy to be applied.
- The experiment on WebArena is extensive, with a comparison of multiple base models and ablations to show improvement of individual components.
summary: The paper proposes a new method for improving LLM-based web agents. The contributions are mainly two folds:

1. Improve alignment between the action and observation space of the web agent and the data on which the LLM has been pre-trained. The authors propose heuristics to: a. Simplify the actions available to the agent and b. Better clean and format the web page observation input for the LLM.

2. New actions and improvement for planning. The authors introduce new `branch` and `prune` actions for altering the plan, as well as prune the web elements included in the historical context.

The authors conduct experiments with WebArena, a web agent benchmark that includes simulation environments across 5 websites.

- The proposed method obtains significant improvement over the base agent design from the original WebArena.
- The proposed method also outperforms the SOTA method on WebArena by 6 points. (Note that the base model is not the same)
- Through ablations, both the action/observation alignment and improvement in planning contribute to the overall bump in performance, with the most gain coming from the action/observation simplification.
- The proposed method can also be combined with another method though the improvement depends on specific agent design. In Step, the proposed method brings improvement compared with Step, though it does not bring further improvement to AgentOccam. When combined with an LLM judge, the authors observe further improvement over AgentOccam.
weaknesses: - The main contribution, and also the main source of improvement, is the simplification of action and observation space. A natural question would be how well these simplifications could be generalized across websites and tasks. Although WebArena contains multiple tasks, it still only contains five websites. To further verify the effectiveness of the proposed method, I would suggest including experiments on other datasets, e.g., Mind2Web, which has more websites.

- Similarly, the alignment could potentially be LLM-dependent. The paper only conducted experiments with GPT-4; it would be better if it could be shown that the method could generalize to different LLMs, which potentially have seen different mixes and formats of training data.

- How to properly represent webpage observation and select the action space has been studied by a few works, both for web agents and web IE, in the past (1) (or, in general, all papers on web agents have touched on this topic). It is appreciated that the authors find a new representation that works very well, but the technical novelty is a bit limited, especially for conferences like ICLR.

1. Zhou Y, Sheng Y, Vo N, Edmonds N, Tata S. Simplified DOM Trees for Transferable Attribute Extraction from the Web.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 2
questions: Please see the cons. Please point out if I have any misunderstandings, and I'm willing to improve my rating if the author could greatly address my concerns.
soundness: 2
strengths: Pros:
- The perspective to improve the capability of web agents is great, which it dives into the action space and observation space. It is a common pathway that may take effect in not only web tasks but also other tasks with complex actions and observations.
- I think the action optimization is inspired, especially for abstracting and generating new actions. From my perspective, it is similar to the tool learning problem and can be effective.
- The improvement in performances seems great, where it demonstrates a huge improvement on WebArena.
summary: This paper proposes a new pathway to improve the performance of LLM-based web agents. Compared with previous works on web agents, it mainly focuses on optimizing the action space and observation space, rather than designing reasoning/planning/memory modules for more complex inferences. The performance on WebArena shows the effectiveness of AgentOccam on web tasks.
weaknesses: Cons:
- The author claims to solve the generalizability across all real-world applications in the abstract. However, the method section seems to just illustrate how to deal with the specific web environment. There are too many 'specifically' and 'in particular' in sections 4.1 and 4.2. Does it mean I need to design different merges manually, where the method is just providing a guide or strategy to conduct the manual process? I think an ideal optimization is to conduct the optimization automatically, even by prompting LLMs. You may provide concrete examples of how your approach could be applied to other domains, or discuss any limitations in generalizability.
- I think a general method to automatically conduct the alignment is necessary, especially for the action space mentioned above. But this work seems to fail to do so.
- I'm confused about the part 'Replaying Observation History Selectively', could you please demonstrate with more details? I think this is an information retrieval process, but I'm aware of how to get relevant information. You may provide a step-by-step example of how the selective replay process works on a sample task. This would help clarify the information retrieval aspect that I'm unsure about. Additionally, you might show more details on how the relevance of information is determined and how this process differs from or improves upon standard information retrieval techniques.

Meta-review: This work introduces AgentOccam, an approach designed to align the action and observation spaces of large language model (LLM) agents for web-based tasks. Specifically, the method features an automated pipeline for pruning web page content and defining a more concise action space, optimizing the agent’s interaction efficiency. The proposed approach achieves impressive performance on the WebArena benchmark and demonstrates generalization capabilities through evaluations in live web environments using WebVoyager.

A notable limitation of this work is its focus on web agents that operate solely on text-based representations of web pages, such as HTML or accessibility trees. While effective in its scope, this constraint may limit its applicability to more visually complex or non-text-dominant web interfaces.

Decision: Accept (Poster)