AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents

Authors: Ke Yang, Yao Liu, Sapana Chaudhary, Rasool Fakoor, Pratik Chaudhari, George Karypis, Huzefa Rangwala
Primary area: foundation or frontier models, including LLMs
Keywords: LLM, Agent, LLM-based Agent, Web Agent, Web Navigation

Abstract:
Autonomy via agents based on large language models (LLMs) that can carry out personalized yet standardized tasks presents a significant opportunity to drive human efficiency. There is an emerging need and interest in automating web tasks  (e.g., booking a hotel for a given date within a budget). Being a practical use case itself, the web agent also serves as an important proof-of-concept example for various agent grounding scenarios, with its success promising advancements in many future applications. Meanwhile, much prior research focuses on handcrafting their web agent strategies (e.g., agent's prompting templates, reflective workflow, role-play and multi-agent systems, search or sampling methods, etc.) and the corresponding in-context examples. However, these custom strategies often struggle with generalizability across all potential real-world applications. On the other hand, there has been limited study on the misalignment between a web agent's observation and action representation, and the data on which the agent's underlying LLM has been pre-trained. This discrepancy is especially notable when LLMs are primarily trained for language completion rather than tasks involving embodied navigation actions and symbolic web elements. In our study, we enhance an LLM-based web agent by simply refining its observation and action space, aligning these more closely with the LLM's capabilities. This approach enables our base agent to significantly outperform previous methods on a wide variety of web tasks. Specifically, on WebArena, a benchmark featuring general-purpose web interaction tasks, our agent AgentOccam surpasses the previous state-of-the-art and concurrent work by 9.8 (+29.4%) and 5.9 (+15.8%) absolute points respectively, and boosts the success rate by 26.6 points (+161%) over similar plain web agents with its observation and action space alignment. Furthermore, on WebVoyager benchmark comprising tasks defined on real-world websites, AgentOccam exceeds the former best agent by 2.4 points (+4.6%) on tasks with deterministic answers. We achieve this without using in-context examples, new agent roles, online feedback or search strategies. AgentOccam's simple design highlights LLMs' impressive zero-shot performance on web tasks, and underlines the critical role of carefully tuning observation and action spaces for LLM-based agents.

Decision: accept (poster)
Reviewer scores: 8, 5, 6

Reviews and discussion:
<thread>
creation date:
1730019294291

thread_type:
official_review

<thread_object>
creator: reviewer_tca3

<content>
summary:
This paper proposes a new pathway to improve the performance of LLM-based web agents. Compared with previous works on web agents, it mainly focuses on optimizing the action space and observation space, rather than designing reasoning/planning/memory modules for more complex inferences. The performance on WebArena shows the effectiveness of AgentOccam on web tasks.

strengths:
Pros:
- The perspective to improve the capability of web agents is great, which it dives into the action space and observation space. It is a common pathway that may take effect in not only web tasks but also other tasks with complex actions and observations.
- I think the action optimization is inspired, especially for abstracting and generating new actions. From my perspective, it is similar to the tool learning problem and can be effective.
- The improvement in performances seems great, where it demonstrates a huge improvement on WebArena.

weaknesses:
Cons:
- The author claims to solve the generalizability across all real-world applications in the abstract. However, the method section seems to just illustrate how to deal with the specific web environment. There are too many 'specifically' and 'in particular' in sections 4.1 and 4.2. Does it mean I need to design different merges manually, where the method is just providing a guide or strategy to conduct the manual process? I think an ideal optimization is to conduct the optimization automatically, even by prompting LLMs. You may provide concrete examples of how your approach could be applied to other domains, or discuss any limitations in generalizability.
- I think a general method to automatically conduct the alignment is necessary, especially for the action space mentioned above. But this work seems to fail to do so.
- I'm confused about the part 'Replaying Observation History Selectively', could you please demonstrate with more details? I think this is an information retrieval process, but I'm aware of how to get relevant information. You may provide a step-by-step example of how the selective replay process works on a sample task. This would help clarify the information retrieval aspect that I'm unsure about. Additionally, you might show more details on how the relevance of information is determined and how this process differs from or improves upon standard information retrieval techniques.

questions:
Please see the cons. Please point out if I have any misunderstandings, and I'm willing to improve my rating if the author could greatly address my concerns.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730628209751

thread_type:
official_review

<thread_object>
creator: reviewer_lc8w

<content>
summary:
The paper proposes a new method for improving LLM-based web agents. The contributions are mainly two folds:

1. Improve alignment between the action and observation space of the web agent and the data on which the LLM has been pre-trained. The authors propose heuristics to: a. Simplify the actions available to the agent and b. Better clean and format the web page observation input for the LLM.

2. New actions and improvement for planning. The authors introduce new `branch` and `prune` actions for altering the plan, as well as prune the web elements included in the historical context.

The authors conduct experiments with WebArena, a web agent benchmark that includes simulation environments across 5 websites.

- The proposed method obtains significant improvement over the base agent design from the original WebArena.
- The proposed method also outperforms the SOTA method on WebArena by 6 points. (Note that the base model is not the same)
- Through ablations, both the action/observation alignment and improvement in planning contribute to the overall bump in performance, with the most gain coming from the action/observation simplification.
- The proposed method can also be combined with another method though the improvement depends on specific agent design. In Step, the proposed method brings improvement compared with Step, though it does not bring further improvement to AgentOccam. When combined with an LLM judge, the authors observe further improvement over AgentOccam.

strengths:
- The overall experiment results are positive, and new SOTA performance was obtained on the WebArena benchmark.
- The proposed design is simple yet effective, so it should be rather easy to be applied.
- The experiment on WebArena is extensive, with a comparison of multiple base models and ablations to show improvement of individual components.

weaknesses:
- The main contribution, and also the main source of improvement, is the simplification of action and observation space. A natural question would be how well these simplifications could be generalized across websites and tasks. Although WebArena contains multiple tasks, it still only contains five websites. To further verify the effectiveness of the proposed method, I would suggest including experiments on other datasets, e.g., Mind2Web, which has more websites.

- Similarly, the alignment could potentially be LLM-dependent. The paper only conducted experiments with GPT-4; it would be better if it could be shown that the method could generalize to different LLMs, which potentially have seen different mixes and formats of training data.

- How to properly represent webpage observation and select the action space has been studied by a few works, both for web agents and web IE, in the past (1) (or, in general, all papers on web agents have touched on this topic). It is appreciated that the authors find a new representation that works very well, but the technical novelty is a bit limited, especially for conferences like ICLR.

1. Zhou Y, Sheng Y, Vo N, Edmonds N, Tata S. Simplified DOM Trees for Transferable Attribute Extraction from the Web.

questions:
1. Are there experiments on more datasets to show the effectiveness of the proposed heuristic beyond WebArena?

2. Are there experiments with other LLMs.

3. If possible, it would be better to include results using the same base LLM. It is okay to report SOTA on the best LLM available. But for fair comparison and a better understanding of the contribution of the proposed method, it would be very helpful to include results using the same LLM as the baselines.

4. I recommend swapping Figure 5 and Table 10 for the ablation results. The numbers in the table is easier to consume.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730691148379

thread_type:
official_review

<thread_object>
creator: reviewer_naff

<content>
summary:
This work focuses on developing a simple generalized framework for LLM-based web agents, allowing them to leverage the strengths of LLMs to get complete web tasks. This framework focuses on aligning LLMs to the web domain and hence converts the problem to that of finding alignments/mappings in observation and action space between web domain and LLMs. As part of that, for action space alignment, they reduce actions to simple set of actions and add few additional "planning" related actions to the set. For observation space alignment, they propose to simplify the web page representation and also focus on selective representation of history for tracking. They show impressive results of the impact of these strategies on WebArena benchmark while also providing ablations to understand the impact of each of the recommended change. Finally, they show how this simple baseline agent can be promisingly combined with existing strategies.

strengths:
The following are the major strengths of the paper:
-  They have focused on the first principles of building a domain-specific agent, i.e., a web agent and hence utilized domain-knowledge to optimized how LLMs can be used in the best way for this domain.
- The ablations presented are very well presented allowing the reader to understand the impact of each of the changes proposed in the paper.

weaknesses:
- The results are shown on WebArena which is a simulated environment. It would be more realistic to see how it performs on WebVoyager which is more realistic in terms of how web behaves.
- The tree representations used for planning needs better explanation. I have to honest that I couldn't grasp it as well as I would want to. While I understand it in principle, presenting it with a more detailed example, even if in Appendix, might help someone like me look into the details of it.
- While the methodology/process presented in this paper of "optimizing/aligning LLMs to the domains when designing agents" is very important and valuable, it is worth noting that the specific proposed changes themselves (understandably) don't generalize to other domains beyond web.
- Minor typo in title of Sec 5 (not a weakness obviously but would be good to fix in the final version).

questions:
- I would be curious to see how the agent performs on other benchmarks, especially WebVoyager. This is especially valuable benchmark as besides being more realistic, it will help answer my next question.
- The ideas presented in this work are extremely similar to another concurrent work on building a WebVoyager SOTA web agent, Agent-E. There is almost a one-to-one mapping between the proposed changes (both action and observational space) in AgentOccam and Agent-E. 
i. The simplifying of action space in AgentOccam is close to the selection of "primitive actions" in Agent-E.
ii. The Planning tree in AgentOccam is close in principle to "hierarchical planning" in Agent-E.
iii. Simplifying Web Page observation in AgentOccam is same in principle to the idea behind "DOM distillation" in Agent-E.
iv. Selective replaying of observation history in AgentOccam is similar in principle to "change observation" in Agent-E.
And both agents are SOTA on different benchmarks, AgentOccam on WebArena and Agent-E on WebVoyager, and hence would be good to have a detailed comparison, both quantitative and qualitative, between them.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732307899119

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Experiments on AgentOccam's Generalizability and Code and Trajectory Logs Update

comment:
We appreciate all reviewers' feedback and recognition of AgentOccam's action and observation space alignment's contributions to the LLM-based web agent community. Here, we respond to the common questions raised by the reviewers. **We have updated the manuscript according to the feedback, and have added code for all experiments and new experiment trajectories to the supplementary materials.**

## AgentOccam is Generalizable to Real-world Websites
We additionally conduct experiments on the general-purpose web task benchmark WebVoyager and compare AgentOccam's performance with the benchmarks' SOTA Agent-E. The details are as follows:

WebVoyager benchmark (https://arxiv.org/abs/2401.13919) compiles web tasks from 15 popular real-world websites. It comprises tasks with two types of user questions: ones with "golden" answers that are definite and time-invariant, and ones with "possible" answers that are either open-ended with multiple potential answers or related to real-time information. We use WebVoyager questions with golden answers to avoid subjective human evaluations and exclude GitHub tasks, as the site's anti-scraping measures frequently cause page loading timeouts. Additionally, due to these measures, GitHub limits interactable elements, which prevents web page proper functionality, and the IP address hosting the agent is at risk of being banned. In contrast, WebArena's simulated GitLab environment mimics GitHub, enabling us to demonstrate AgentOccam's performance on similar tasks using existing results. In a nutshell, we have 129 tasks from WebVoyager with definite golden answers across 13 real-world web environments, including Allrecipes, Amazon, Apple, ArXiv, BBC News, Booking, Cambridge Dictionary, Coursera, ESPN, Google Map, Google Search, Huggingface, and Wolfram Alpha.

Our baseline on WebVoyager is the concurrent work Agent-E (https://arxiv.org/abs/2407.13032), which introduces several architectural improvements like the planner-browser-reflector agent architecture and the browser's document object model (DOM) selection. It achieves the previous SOTA on the full WebVoyager with the assessments done by humans. As they didn't report agent trajectory logs, we replicate their work on the WebVoyager subset introduced in the above paragraph, evaluated with the same definite-answer-based hard-coded evaluators as ours. We run each task one time for both agents. The detailed success rates (SRs) are as follows with the code and trajectory logs attached as the supplementary materials:

| Website (Task Number)               | Agent-E SR | AgentOccam SR |
|---------------------------------------|--------------------|-----------------------|
| **Allrecipes (4)**                    | 75.00%             | 50.00%               |
| **Amazon (1)**                        | 0.00%              | 0.00%                |
| **Apple (7)**                         | 57.14%             | 28.57%               |
| **ArXiv (16)**                        | 50.00%             | 43.75%               |
| **BBC News (2)**                      | 0.00%              | 0.00%                |
| **Booking (2)**                       | 50.00%             | 100.00%              |
| **Cambridge Dictionary (9)**          | 55.56%             | 88.89%               |
| **Coursera (2)**                      | 50.00%             | 50.00%               |
| **ESPN (10)**                         | 40.00%             | 50.00%               |
| **Google Map (9)**                    | 33.33%             | 44.44%               |
| **Google Search (16)**                | 62.50%             | 81.25%               |
| **Huggingface (17)**                  | 17.65%             | 29.41%               |
| **Wolfram Alpha (34)**                | 73.53%             | 61.76%               |
| **Overall (129)**                     | 51.9%              | 54.3%                |

Based on the results above, AgentOccam performs better than Agent-E on the WebVoyager's definite-answer-subset. Specifically, both agents have their specificities. We find that Agent-E makes an edge on websites like Wolfram Alpha with more delicate plans issued by its "planner," and AgentOccam outperforms it on websites like Google Search and Hugging Face with more accurate task interpretation and information retrieval. The impressive results across 10+ websites and many types of tasks can show AgentOccam's generalizability in the web domain.
</content>
</thread_object>

</thread>
<thread>
creation date:
1733176067973

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
We express our gratitude to all reviewers for their feedback, which has enriched our work. We are delighted to see that the reviewers have identified several strengths in our paper, including:

* The paper proposes a simple yet effective LLM-based web agent (NaFf, LC8w, Tca3);
* The idea of aligning the observation and action representation of web tasks with LLM capabilities is crucial and inspiring (NaFf, Tca3), offering a new pathway that could extend to other complex tasks (Tca3);
* The results show impressive improvements over the plain base agent and the state-of-the-art on the WebArena benchmark (NaFf, LC8w, Tca3);
* The ablation studies are comprehensive (NaFf, LC8w);
* Our simple baseline agent can be promisingly combined with existing strategies, e.g., LLM-as-a-judge (NaFf, LC8w).

Upon reviewers' suggestions, we have included additional experiments that show our web agent is generalizable to real-world websites and different LLM families. We have also added code for all experiments and new experiment trajectories to the supplementary materials.

We are grateful for the opportunity to discuss our work with the reviewers and are enthusiastic about the potential impacts of our contributions to the field. Thank you for your encouragement and for recognizing our efforts.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This work introduces AgentOccam, an approach designed to align the action and observation spaces of large language model (LLM) agents for web-based tasks. Specifically, the method features an automated pipeline for pruning web page content and defining a more concise action space, optimizing the agent’s interaction efficiency. The proposed approach achieves impressive performance on the WebArena benchmark and demonstrates generalization capabilities through evaluations in live web environments using WebVoyager.

A notable limitation of this work is its focus on web agents that operate solely on text-based representations of web pages, such as HTML or accessibility trees. While effective in its scope, this constraint may limit its applicability to more visually complex or non-text-dominant web interfaces.