R a DA : Retrieval-augmented Web Agent Planning with LLM s (2024)

Abstract: Agents powered by large language models (LLMs) inherit important limitations, such as the restricted context length, dependency on human-engineered exemplars (e.g., for task decomposition), and insufficient generalization. To address these challenges, we propose RaDA, a novel planning method for Web agents that does not require manual exemplars, efficiently leverages the LLMs’ context, and enhances generalization. RaDA disentangles planning into two stages: for a new given task, during Retrieval-augmented Task Decomposition (RaD), it decomposes tasks into high-level subtasks; next, during Retrieval-augmented Action Generation (RaA), it traverses the trajectory obtained with RaD to iteratively synthesize actions based on dynamically retrieved exemplars. We compare RaDA with strong baselines covering a broad space of design choices, using both GPT-3.5 and GPT-4 as backbones; and we find consistent improvements over previous SOTA in two challenging benchmarks, CompWoB and Mind2Web, covering settings with different complexities. We show the contributions of RaDA via ablation studies and qualitative analysis; and we discuss the structural benefits of our more compositional design.

## introduction the adaptability of general-purpose, large language models (llms) is a key driver of their growing application across diverse scenarios. central to this adaptability is in-context-learning (icl), which enables llms to perform new tasks based on demonstrations, bypassing the need for a dedicated finetuning phase @xcite . this capability is crucial in scenarios involving llms deployed as agents in interactive environments, where finetuning llm-based agents with extensive data across † corresponding author. all possible interactions is impractical @xcite . however, despite their promise, existing methods of llm task adaptation face significant limitations. first, icl-based methods incur a significant cost of prompt engineering. existing approaches rely on exemplars which must be carefully authored by human annotators, aiming to fully detail the target task. second, extensive research has shown that llms exhibit a high degree of sensitivity to prompt format and the choice of exemplars, making prompt optimization very challenging @xcite . an increasingly prominent line of work proposes the dynamical retrieval of exemplars for icl conferring two crucial advantages: 1) it automates the prompt engineering process, reducing the cost of exemplar annotation. 2) it aligns exemplars to the task context, enhancing the robustness of exemplar selection. these advantages lead to performance improvements over human-engineered prompts, as empirically demonstrated in a wide range of experimental results @xcite @xcite . in the context of llms-as-agents, the closest approach to ours is synapse @xcite , which retrieves trajectory demonstrations based on semantic retrieval, and leverages them as replacements to human-written prompts. in this work, we consider synapse as a baseline for pure retrievalaugmented action generation (raa). however, raa such as synapse seeks to infer the entire action plan for the current task, based on similar entire trajectories of previously seen tasks. this process would fail to generalize when the target task is: 1) unseen, i.e., outside of the corpus of previously seen exemplars, and 2) complex, i.e., compositions of more atomic tasks. to demonstrate these limitations, we will experimentally show how the performance of existing state-of-art approaches strongly degrades in complex or unseen scenarios. we hypothesize that this failure to generalize can be attributed to the lack of compositional consistency observed in llms, regardless of size @xcite @xcite . that is, llms cannot abstract and recombine learned behaviors, such that having llms directly transform a task description into an action sequence cannot generalize for unseen and complex tasks. as a solution, we disentangle planning into decomposition into subtasks, then composing retrieved actions in a two-stage process: first, we generate a high-level plan based on a principle of subtask decomposition via retrieval-augmented task decomposition (rad); second, we perform grounded action generation for each of the subtasks via raa. our approach is closely inspired by two fundamental concepts in human cognitive processing: 1) the ability to disentangle planning and behavior grounding through the use of abstraction @xcite , and 2) the efficient reuse of memory through retrieval, to adapt to novel situations through the effective utilization of existing knowledge @xcite . by leveraging contextually retrieved subtask knowledge in the form of exemplars, rada efficiently leverages the llms' context in an adaptive manner, while alleviating the need for manually annotated few-shot exemplars. we evaluate our method on two challenging benchmarks for interactive web environments, covering a range of complex scenarios, while comparing it with strong method human annotation effort input length compositional generalization decomp high short ✓ rci high short ✗ synapse none long ✗ ours none short ✓ our contributions can be summarized as follows: ## related work this section classifies baseline models ( ## conclusion we propose rada, a planning method which enhances the compositional generalization of llm agents for web tasks. rada disentangles planning into two components, retrieval-augmented task decomposition (rad) which decomposes tasks into high-level subtasks, and retrievalaugmented action generation (raa) which enables the grounded execution of subtasks without the need for manually authored few-shot prompts. we show through experiments on the compwob and mind2web tasks that rada enhances the compositional generalization of llm agents, significantly outperforming competitive baselines.