RaDA: Retrieval-augmented Web Agent Planning with LLMs (2024)

Abstract: Agents powered by large language models (LLMs) inherit important limitations, such as the restricted context length, dependency on human-engineered exemplars (e.g., for task decomposition), and insufficient generalization. To address these challenges, we propose RaDA, a novel planning method for Web agents that does not require manual exemplars, efficiently leverages the LLMs’ context, and enhances generalization. RaDA disentangles planning into two stages: for a new given task, during Retrieval-augmented Task Decomposition (RaD), it decomposes tasks into high-level subtasks; next, during Retrieval-augmented Action Generation (RaA), it traverses the trajectory obtained with RaD to iteratively synthesize actions based on dynamically retrieved exemplars. We compare RaDA with strong baselines covering a broad space of design choices, using both GPT-3.5 and GPT-4 as backbones; and we find consistent improvements over previous SOTA in two challenging benchmarks, CompWoB and Mind2Web, covering settings with different complexities. We show the contributions of RaDA via ablation studies and qualitative analysis; and we discuss the structural benefits of our more compositional design.

heading: 1 Introduction
text: The adaptability of general-purpose, large language models (LLMs) is a key driver of their growing application across diverse scenarios. Central to this adaptability is in-context-learning (ICL), which enables LLMs to perform new tasks based on demonstrations, bypassing the need for a dedicated finetuning phase (Brown et al., 2020). This capability is crucial in scenarios involving LLMs deployed as agents in interactive environments, where finetuning LLM-based agents with extensive data across
†Corresponding author.
all possible interactions is impractical (Yao et al., 2023; Carta et al., 2023).
However, despite their promise, existing methods of LLM task adaptation face significant limitations. First, ICL-based methods incur a significant cost of prompt engineering. Existing approaches rely on exemplars which must be carefully authored by human annotators, aiming to fully detail the target task. Second, extensive research has shown that LLMs exhibit a high degree of sensitivity to prompt format and the choice of exemplars, making prompt optimization very challenging (Madaan and Yazdanbakhsh, 2022; Yang et al., 2024).
An increasingly prominent line of work proposes the dynamical retrieval of exemplars for ICL conferring two crucial advantages: 1) It automates the prompt engineering process, reducing the cost of exemplar annotation. 2) It aligns exemplars to the task context, enhancing the robustness of exemplar selection. These advantages lead to performance improvements over human-engineered prompts, as
13511
empirically demonstrated in a wide range of experimental results (Khattab et al., 2022; Luo et al., 2023; Li et al., 2023).
In the context of LLMs-as-agents, the closest approach to ours is Synapse (Zheng et al., 2024b), which retrieves trajectory demonstrations based on semantic retrieval, and leverages them as replacements to human-written prompts. In this work, we consider Synapse as a baseline for pure Retrievalaugmented Action Generation (RaA).
However, RaA such as Synapse seeks to infer the entire action plan for the current task, based on similar entire trajectories of previously seen tasks. This process would fail to generalize when the target task is: 1) Unseen, i.e., outside of the corpus of previously seen exemplars, and 2) Complex, i.e., compositions of more atomic tasks. To demonstrate these limitations, we will experimentally show how the performance of existing state-of-art approaches strongly degrades in complex or unseen scenarios.
We hypothesize that this failure to generalize can be attributed to the lack of compositional consistency observed in LLMs, regardless of size (Dziri et al., 2023; Zheng et al., 2024a; Chen et al., 2024). That is, LLMs cannot abstract and recombine learned behaviors, such that having LLMs directly transform a task description into an action sequence cannot generalize for unseen and complex tasks. As a solution, we disentangle planning into decomposition into subtasks, then composing retrieved actions in a two-stage process: first, we generate a high-level plan based on a principle of subtask decomposition via Retrieval-augmented Task Decomposition (RaD); second, we perform grounded action generation for each of the subtasks via RaA.
Our approach is closely inspired by two fundamental concepts in human cognitive processing: 1) the ability to disentangle planning and behavior grounding through the use of abstraction (Lachmy et al., 2022), and 2) the efficient reuse of memory through retrieval, to adapt to novel situations through the effective utilization of existing knowledge (Zhao et al., 2022; Sumers et al., 2023).
By leveraging contextually retrieved subtask knowledge in the form of exemplars, RaDA efficiently leverages the LLMs’ context in an adaptive manner, while alleviating the need for manually annotated few-shot exemplars. We evaluate our method on two challenging benchmarks for interactive Web environments, covering a range of complex scenarios, while comparing it with strong
baselines covering a broad space of design choices. Our contributions can be summarized as follows:
• We propose RaDA, a novel retrievalaugmented planning method for LLMs, which disentangles planning and action grounding to enhance compositional generalization.1
• We show that our method improves generalization in challenging scenarios, significantly outperforming competitive baselines on both the CompWoB interactive tasks, and the Mind2Web dataset.

heading: 2 Related Work
text: This section classifies baseline models (Table 1) based on two axes: their target focus and the degree of human supervision involved.

heading: 2.1 Decomposition and Planning in LLM
text: Decomposition Prior literature has explored LLM prompting which aims to induce a modular reasoning structure based on decomposition. Leastto-Most prompting (Zhou et al., 2023) demonstrates that breaking down in subproblems aids generalization in symbolic manipulation, compositional generalization, and math word problems (Cobbe et al., 2021). DecomP (Khot et al., 2023) further enables diverse decomposition structures, including recursion and other non-linear decomposition structures, extending the applicability to long-context multi-hop QA. Wang et al. (2023) show that decompositional prompting is effective for math reasoning even in zero-shot settings, and
1Code is available at: https://github.com/ldilab/ RaDA
Drozdov et al. (2022) shows that combining decomposed subproblems with dynamic exemplar selection enhances performance on compositional generalization in simple NLU tasks (Keysers et al., 2020).
RaDA tackles compositional generalization, by leveraging exemplar retrieval of actions per dynamically defined subtask, which allows the agent to flexibly compose grounded plans that can generalize to new scenarios. This is closest to DecomP in Table 1 (column 3), which demonstrates compositional generalization on a multi-hop reading comprehension task. Differently from DecomP, RaDA does not require extensive manual exemplars, and thus can be evaluated on interactive tasks.
Action Generation for LLM Agents LLM have been leveraged as agents in diverse scenarios, for their ability to adapt and ground actions to environment observations, such as robotics (Ahn et al., 2022; Huang et al., 2022). Building on these, ReAct (Yao et al., 2023) proposes to interleave the generation of both reasoning traces and task-specific actions, enabling the LLM to track and update action plans for interactive tasks in virtual environments. An extension of ReAct, Reflexion (Shinn et al., 2023), adds a trial-level memory mechanism, enabling the agent to learn from multiple attempts of the same task, through trial-and-error.
For our target application of LLM-based agents for Web tasks, the following baselines have studied the use of LLMs with human-engineered exemplars. RCI (Kim et al., 2023) leverages a process of iterative self-refinement, to enhance grounding by repairing errors through LLM’s self-critique. AdaPlanner (Sun et al., 2023) leverages an adaptive planning mechanism, which utilizes few-shot exemplars to revise a plan in response to errors.
RaDA is closest to Synapse (Zheng et al., 2024b) in Table 1 (column 1) as neither requires humanengineered exemplars, instead obtaining exemplars through retrieval, where an exemplar retrieval corpus can be constructed by collecting trajectories with little to no annotation effort. However, Synapse requires the retrieval of whole-trajectory exemplars, which often do not exist in memory; and even when they do, can become too lengthy for LLMs’ context. In contrast, RaDA (Table 1, column 2) can leverage partial trajectories via RaD, making more efficient use of the context and allowing for improved compositional generalization. Comparative discussion with respect to human an-
notation follows in the next section.

heading: 2.2 Human Annotation
text: For supervising the ICL process, early work explored the use of human-engineered, task-specific exemplars (Wei et al., 2022; Liu et al., 2023). However, this approach not only incurs high annotation cost we aim to avoid, but also exhibits various types of sensitivity to the specification of prompts, such as the format of the prompt text (Min et al., 2022; Kojima et al., 2022; Yang et al., 2024; Wei et al., 2023; Madaan and Yazdanbakhsh, 2022), or the order of the exemplars presented in the few-shot prompt (Zhao et al., 2021; Lu et al., 2022),
Similarly, methods such as DecomP can become too costly and ineffective to annotate, as it requires detailed manual demonstrations of problem decomposition and subproblem solution. In contrast, our method disentangles prompts into high-level subtask planning and subtask-level demonstrations, alleviating the need for annotating exemplars that demonstrate subtask composition in detail, and enhancing robustness in compositional generalization. Simultaneously, the same process improves context length efficiency, by removing the need for lengthy exemplars for the overall task.

heading: 3 Method

heading: 3.1 Compositionality Challenge in Planning
text: Before describing our method in detail, we first provide a description of standard LLM-based agents, within which we highlight the challenge of compositional generalization.
Language Model Agents Formally, given a task instruction T , an LLM agent π is configured with the prompt P , with the goal of generating a plan Γ for the task, as follows:
ΓT “ ta1, ..., atu Ð πp¨|P, T q (1) where the plan consists of a sequence of actions to be executed in the environment Env, as
st`1, rt`1 “ Envpst, atq (2) and task success is measured by the episodic reward function r : S ˆ A Ñ t0, 1u. Compositional Consistency Existing planning methods which are encapsulated by Eq.1, including RCI and Synapse, amalgamate both overall task planning and the generation of environmentspecific actions, as a simultaneous process. This
Algorithm 1 RaDA Input: task T , environment Env, retriever ϕ, and πD , πQT , πV , representing LLM-based RaD, RaA, and subtask completion
verifier, respectively. 1: s ÐÝ Env.reset(T) 2: history ÐÝ tu 3: C ÐÝ tu 4: Γ ÐÝ tu 5: h ÐÝ 5 6: while Env is not terminated do 7: Γ “ tγ1, ..., γnu ÐÝ πDp¨|ϕpT,Eq, s, T q Ź Subtask Decomposition (RaD) 8: while C ‰ Γ do 9: QT ÐÝ πV ps, T,Γ, C, historyr´h :sq Ź Subtask-compositional Query 10: ta1, ..., atQT u ÐÝ πQT p¨|ϕpQT , Eq, s, QT , historyr´h :sq Ź Action generation (RaA) 11: for a in ta1, ..., atQT u do 12: s, r, info ÐÝ Env.step(a) 13: history ÐÝ history Y tps, aqu 14: end for 15: Verified ÐÝ πV ps, T,Γ, C, historyr´h :sq Ź Subtask Completion Verification 16: for γ in Verified do 17: C ÐÝ C Y tγu 18: end for 19: end while 20: end while
is effective when task T as a whole maps to action sequences in the memory, but would not generalize for tasks that are complex and novel compositions of existing behaviors, unless LLMs can abstract and recombine learned behaviors (or, compositional consistency), which is known to lack in LLMs regardless of model size (Chen et al., 2024). We make consistent observations and replace LLM inference with compositional retrieval to overcome.

heading: 3.2 Proposed: Disentangling Planning as RaD+RaA
text: Our proposed approach aims to tackle this challenge, through a principle of disentangling planning into hierarchical components: 1) A decompositional planner which breaks down a task into a set of high-level subtasks and tracks their completion status, and 2) retrieval-augmented, dynamic subtask functions which operationalize the execution of subtasks, converting them into action sequences.
Rather than inferring planning and action generation for an entire task T from a singular prompt P , we decompose unseen or complex tasks into seen subtasks and compose their efficient and generalization execution.

heading: 3.2.1 Compositional Exemplar Memory
text: Formally, the core component of our method is a nuanced, adaptive retrieval mechanism, with the basic unit of this retrieval process defined as the subtask, denoted by γ. A query for subtask retrieval is QT , or, subtask-compositional query, defined as below.
Given a set of subtasks Γ “ tγ1, γ2, . . . , γnu, which decomposes the task T , a subtaskcompositional query is formulated as:
QT “ γ1 ‘ γ2 ‘ . . . ‘ γm (3) where ‘ denotes concatenation, and γj represents the j-th subtask of the m selected subtasks for the query composition. RaDA composes such queries to search its task memory, leveraging the retrieved exemplars to ground its action generation. The formulation in Eq.3 enables effective search for a flexible array of composed task descriptions, ranging from focused, single-subtask queries to complex, multi-subtask compositions, which we further detail in Sec.3.2.3.
For the retrieval model, we leverage dense retrieval using pre-trained sentence embeddings (Karpukhin et al., 2020). Given a query Q, the retriever ϕ produces a ranking over a corpus E of encoded exemplars te1, e2, . . . , elu as follows:
tsimpQ, eq | e P Eu Ð ϕpQ,Eq (4) Note that in Zheng et al. (2024b), the query Q is predetermined to be the full textual description of task T . Our distinction is leveraging ϕ with a dynamically composed query QT , which is adaptive to the current state of the plan.
After ranking, the retriever outputs the top-k exemplars, where k is a hyperparameter set to 3. For the retriever setup, we follow Zheng et al. (2024b). We use cosine similarity as the similarity measure, and leverage pre-trained embeddings to encode the
task description, complemented by additional inputs of state and task metadata for CompWoB and Mind2Web, respectively. The retrieval corpus is constructed by successful trajectories from 48 base tasks of MiniWoB, and for Mind2Web, trajectories in the training set are used.
In RaDA, ϕ is flexibly utilized in two ways: First, with T as the query during the initial decomposition stage (RaD; Eq.6), and second, using the subtask-compositional query QT , during the action generation stage (RaA; Eq.7), which we describe next.

heading: 3.2.2 RaD: Retrieval-augmented Subtask Decomposition
text: The first step of RaDA is Retrieval-augmented Subtask Decomposition (RaD), formalized as follows. We define a plan ΓT as a collection of subtasks, generated by a decompositional planner πD, as:
ΓT “ tγ1, ..., γnu Ð πDp¨|PπD , T q (5)
Importantly, the planner’s focus is on decomposition, and ΓT can be ordered or unordered. To construct the decomposition prompt PπD , we utilize a basic instructional prompt P IπD
2 in conjunction with contextualization using retrieved exemplars, as:
PπD “ rP IπD , ϕpT,Eqs. (6)
This process leverages the exemplars retrieved using the overall task T as the search query, which facilitates the refinement the planner’s subtask generation on grounded knowledge of the task.
Subtask Completion Verification Once the decomposed subtask set Γ is generated, we leverage a subtask completion verifier to track the progression of the subtasks towards completion, maintaining a set C of completed subtasks. The verifier is implemented using an instruction-prompted LLM, which takes the observation and actions of the last k steps, plus the remaining subtasks Γ ´ C as input, and outputs the completed subtasks, if any have been completed by the latest action. If a subtask is completed, γ is added to C.

heading: 3.2.3 RaA: Retrieval-augmented Action Generation
text: Following the subtask decomposition by RaD, the grounded action generation step, RaA, proceeds to
2See Appendix A.2 for the full prompts.
adaptively generate the grounded action sequence, based on the current stage of the overall plan.
Our subtask-level formulation makes this process intuitive, through the definition of a subtask function πQT , as an LLM whose intended functionality is 1) represented by the subtask-compositional query QT , and is 2) implemented by the exemplars retrieved via ϕpQT , Eq.
Formally, RaA defines πQT , which generates the grounded action sequence as follows3:
ta1, ..., atQT u Ð πQT p¨|ϕpQT , Eqq (7)

heading: 3.2.4 RaDA
text: We demonstrate the full step-by-step operation of RaDA in Alg.1. Given a task T , RaDA first proceeds with RaD, decomposing the task into the set of subtasks ΓT (line 7). Next, RaDA iteratively performs subtask-level action generation through RaA, operating over the subtask set. At each iteration, the subtask-compositional query QT is reformulated, via the verifier πV which verifies the completion of subtasks γ (line 15), before generating QT (line 9). QT can range from a single task, to encompassing all remaining subtasks. RaDA iterates over the subtasks, until all subtasks are completed (line 8).

heading: 4 Experiments

heading: 4.1 Benchmarks
text: To validate the effectiveness of RaDA, we target two web task automation environments with varying compositional characteristics. First, we evaluate on CompWoB, where tasks are compositions of other web tasks, and thus task decomposition is likely to lead to a set of clear subtasks. Second, to further test the generality of our approach, we also evaluate on Mind2Web, a challenging dataset of human-written tasks on real websites, where tasks are not explicitly compositional, making decomposition into subtasks more complex.
CompWoB CompWoB (Furuta et al., 2023) is an interactive environment which modifies the widelystudied MiniWoB (Shi et al., 2017) benchmark for web and computer task automation, to test the compositional generalization capability of agents. The tasks involve elemental computer interactions such as clicking, typing, form-filling, handling popup
3An instructional prompt is also used as in Eq.6, which we omit for notational clarity. See Appendix A.2 for the full prompts.
messages, and login. Furuta et al. (2023) categorizes 65 base tasks from the original MiniWoB (Shi et al., 2017) by task complexity, based on the performance of existing LLM agents, into easy, medium, and hard categories.
The 50 compositional tasks in CompWoB are created by systematically combining 2 to 8 base tasks from the easy category, with different levels of overall task complexity, divided into five categories: two-way tasks (20), three-way tasks (10), n-way tasks (5), transition tasks (5), and easymedium two-way tasks (10). In n-way tasks, 4 to 8 tasks are combined, and in transition tasks, an explicit page transition is implemented, e.g. transitioning from login form to the email browser. The easy-medium two-way tasks incorporate base tasks from the medium category.
In CompWoB, the model input is the state st P S is the raw HTML, and the output actions at are generated for a programmatic action space A consisting of the form, action_type(ID, text), where action_type is one of several possible function types, and ID indicates a unique identifier of an HTML element, where function types are click, move, and type. The additional text input is utilized when the action is type. We report the standard metric of the success rate of the agent in accomplishing tasks.
Mind2Web Mind2Web (Deng et al., 2023) is a dataset of crowdsourced human interaction demonstrations on a diverse set of real websites which are dynamic and complex. Mind2Web targets realistic tasks that require sophisticated interactions with websites, often requiring a large number of steps to complete. The tasks cover diver domains in Travel, Shopping, Service, Entertainment, and Information.
Similar to CompWoB, the model input is the state st P S consisting of the raw HTML from the dataset, but necessarily reduced using a retrieval model, into a smaller set of top-ranked candidate HTML elements, to handle the massive scale of real-world webpages. The output actions use the same programmatic action space of the form, action_type(ID, text), where action_type in Mind2Web, are click, select, and type.
To focus on the evaluation of RaDA on compositional consistency, we select a subset of the challenging Cross-Domain split of Mind2Web, ranked based on compositional task complexity. Task complexity is measured by the number of gold action
steps in the solution. We report the following standard metrics used for this task: element accuracy, measuring the correctness of the selected ID, operation F1, predicting the correctness of the chosen action_type, and the success rate for each step in a task, where success indicates both the element and operation selection are correct. For each task, metrics are averaged over the steps of a task, and the macro average across all tasks is reported.

heading: 4.2 Baselines
text: RCI Recursive Criticism and Improvement (Kim et al., 2023) first samples an initial plan consisting of low-level actions, which is refined once using LLM self-criticism to enhance grounding, Next, before the agent executes each action in the plan, multiple iterations of self-refinement is utilized to improve the grounding of the action. RCI is the SOTA LLM method reported in (Furuta et al., 2023).
Synapse Synapse combines a trajectory-level formulation of planning with retrieved trajectory exemplars, to enhance grounding of the overall plan, based on similar experiences stored in an exemplar memory. For the retrieval memory, exemplars from the 48 base tasks of MiniWoB are used, and in Mind2Web, the same training set trajectories are used as the exemplar memory for retrieval, both as in RaDA.
AdaPlanner We additionally compare with approaches which focus on post-hoc error repair. AdaPlanner (Sun et al., 2023) is a few-shot prompting method which leverages a post-hoc form of plan grounding, by re-planning in response to encountered failures from the environment.
MindAct As the original RCI and AdaPlanner are not designed to be evaluated on Mind2Web, and cannot be easily applied without annotating new few-shot exemplars for the dataset, we compare with the closest equivalent few-shot prompted approach from Deng et al. (2023), as well as Synapse, which is evaluated on Mind2Web. MindAct, the baseline agent from Mind2Web, utilizes a multiplechoice formulation of action generation, performing fine-grained reasoning for grounded action generation using few-shot prompts.

heading: 4.3 Models
text: RaDA Our full model, RaDA, uses both retrievalaugmented plan generation (RaD) and retrievalaugmented subtask grounding (RaA), to enable
compositional generalization without manual fewshot exemplars. All LLM modules are implemented using instructional prompts, shown in Appendix A.2, and all subtask functions are defined using retrieved exemplars. In implementation, RaDA utilizes a decomposition trigger to initiate decomposition. In CompWoB, decomposition behavior is always triggered, and in Mind2Web, decomposition behavior is triggered if the task length exceeds the minimum length exemplar by 50%. Finally, in CompWoB, the subtask-compositional query QT is directly outputted by πV as a single subtask, and in Mind2Web, QT encompasses all remaining subtasks after verification.
RaA (RaD ablation) On CompWoB, we further evaluate variants of RaDA, to understand the effectiveness of each proposed component. First, we test RaA, which partially ablates RaD, and does not use a subtask-decompositional planner to disentangle subtask-level planning from grounded action generation. In this variant, subtask decomposition is utilized only to generate the fine-grained retrieval queries for exemplars. For a fair comparison, we design RaA such that the overall set of exemplars consumed by RaA and RaDA are identical. The retrieved exemplars are simultaneously concatenated
4For each of the WoB tasks we perform multiple runs with different seeds, for a total of 10 runs per task.
into the input context, such that action grounding can leverage retrieved exemplars, albeit without a disentangled notion of subtask progress.
RaD (RaA ablation) We further evaluate RaD, which ablates RaA, thereby removing the ability to retrieve exemplars based on a dynamically generated query. In this variant, we utilize a static set of few-shot exemplars which replace the dynamically retrieved exemplars from RaDA. We choose 5 exemplars consisting of three easy (click-button, click-link, click-dialog), 1 medium (search-engine), and 1 hard (choose-date) tasks according to the task classification of CompWoB.

heading: 5 Results

heading: 5.1 Results on CompWoB
text: In Table 2, we report the results on the CompWoB original and reverse tasks. The latter provides upside down instructions (e.g. do A,B,C Ñ do B and
5Due to high API costs, we compute the performance on the full set (912 examples), using the official logs released by (Zheng et al., 2024b), using an oracle decomposition trigger.
C, after doing A), reflecting the inherent ambiguity in real-world web instructions. Experimental results from Furuta et al. (2023) show that these simple permutations significantly degrade the performance of LLM agents.
Compared to strong baselines on CompWoB, RaDA outperforms all compared models significantly, in both the original and reverse settings. We also report the results of the two ablations, RaA, which utilizes subtask-level exemplar retrieval for action generation, but grounds all steps of the task plan simultaneously, and RaD, which utilizes a decompositional planner, but with a fixed set of exemplars. Compared to the 63.6%{49.0% accuracy of RaDA, both RaA and RaD show much lower accuracies, of 38.0%{23.8% and 50.0%{37.8% respectively, indicating that subtask decomposition and dynamic exemplar retrieval are complementary. Collectively, these findings substantiate our hypothesis that RaDA’s fusion of high-level planning with retrieval-augmented grounding enhances compositional generalization. While baseline models falter when confronted with the compositional task of CompWoB, particularly evident in the reverse setting, and with GPT-3.5, ours consistently demonstrates robust performance across all scenarios, irrespective of model scale.
Furthermore, we evaluate RaDA using the more powerful GPT-4, and observe that performance scales with more performant LLMs, achieving a new state-of-the-art result for LLMs on CompWoB.

heading: 5.2 Results on Mind2Web
text: In Tables 3 and 4, we report the results on the Mind2Web dataset. Our results on the complex subset of Mind2Web reveal that this subset of Mind2Web poses a challenge for the original Synapse method, as shown by the drop in performance (32.8% Ñ 24.73% for element accuracy, 29.33% Ñ 21.34% for step success rate). This indicates that even for state-of-the-art models, compositional generalization remains challenging.
In contrast, RaDA shows strong performance on both the full and the complex subset, outperforming Synapse significantly on all metrics. We further report the results on GPT-4, and the results are consistent with CompWoB, indicating that our method scales well with more performant LLMs in diverse scenarios.

heading: 6 Analysis

heading: 6.1 CompWob Success Rate by Compositional Task Type
text: To provide a better understanding of the performance gains of RaDA on CompWoB, we present a breakdown of CompWoB results by the type of compositional task in Table 5. We first observe that on compositional tasks such as n-way, transition, and easy-medium, existing approaches perform extremely poorly, showing the challenge of compositional generalization. In the case of vanilla Synapse, the reverse setting of CompWoB significantly effects performance further, indicating that exemplar retrieval alone does not solve compositional generalization.
In contrast, RaDA achieves a consistent and large improvement over baselines on all compositional task types, indicating that its disentangled structure enables better compositional generalization. Next, compared to the RaA variant, which has the advantage of a longer context length, but ablates the disentangled structure, RaDA performs better on 8 of the 10 tasks across the original and reverse orders, while being comparable on the remaining two. Similarly, compared to RaD, which ablates the dynamic retrieval of exemplars, it performs better on 7 of the 10 tasks. Finally, the GPT-4 results show that RaDA’s performance scales consistently with more performant LLMs.

heading: 6.2 Context-token Efficiency
text: To validate the efficiency of RaDA’s exemplar retrieval, we measure the context-token efficiency per LLM inference call, as a main driver of LLM inference cost is the quadratic scaling of computation with respect to input context length. We compare token efficiency of RaA and RaDA, measuring the per-call average and maximum numbers of: tokens comprising 1) the exemplars and 2) the entire prompt.
As Table 6 shows, an LLM inference call in RaDA requires on average less than half the amount of tokens, which translates to a 4x computational efficiency. Furthermore, the lengthy input context of RaA due to the concatenation of exemplars results in cases that exceed the 4096 context token limit of the baseline turbo model, thus requiring the usage of gpt-3.5-turbo-16k-0613 (16385 token limit) to accommodate the longer context.
Additionally, since RaDA performs LLM inference call for each subtask, we also compute the average total number of tokens consumed per task. These are 4582.31 / 7285.76, for RaA and RaDA, respectively. While the overall tokens seen is about 2x6, due to the quadratic scaling of transformer computations, RADA is still more computationally efficient due to shorter exemplars in each inference call, while achieving significantly higher performance. These observations lend further support to our claim, that decomposing tasks leads to more efficient and performant in-context learning for LLMs.
6Note that both RaA and RaDA are given access to the same overall set of exemplars. The 2x difference arises from a further processing for RaA which removes redundant exemplars, as RaA concatenates all exemplars together.

heading: 6.3 Complexity of Tasks in Mind2Web
text: In Fig. 2, we show that the distribution of the number of action steps required in the solution, which directly correlate to the complexity of the task (Deng et al., 2023), is skewed towards simpler tasks. For example, the longest task in the crossdomain split requires 26 separate actions to solve correctly, whereas the majority of tasks actually require less than 10 steps, making it likely that a single retrieved exemplar can sufficiently demonstrate a solution. The complex subset in Table.4 corresponds to the gold columns of the figure. Our results demonstrate that state-of-the-art models do not yet generalize well in compositional scenarios, and that RaDA’s retrieval-augmented task decomposition and action generation makes meaningful progress towards compositional generalization.

heading: 7 Conclusion
text: We propose RaDA, a planning method which enhances the compositional generalization of LLM agents for Web tasks. RaDA disentangles planning into two components, Retrieval-augmented Task Decomposition (RaD) which decomposes tasks into high-level subtasks, and Retrievalaugmented Action Generation (RaA) which enables the grounded execution of subtasks without the need for manually authored few-shot prompts. We show through experiments on the CompWoB and Mind2Web tasks that RaDA enhances the compositional generalization of LLM agents, significantly outperforming competitive baselines.
Limitations
A limitation of our method is that we rely on a sufficiently good off-the-shelf retrieval model, and our work does not focus on exploring possibilities for using other various types of pre-trained retrievers, of which there are many.
While we did not consider retriever finetuning in this work, future research could consider an overall framework which finetunes the exemplar retriever in tandem with LLM agents.
Another limitation of our method is that we leverage existing trajectories provided by the environments studied, as the corpora for exemplar retrieval. While the cost of collecting trajectories, as opposed to annotating prompt exemplars, is generally much lower, in some tasks human exemplars may be effective in bootstrapping the trajectory collection process. Finally, while leveraging existing exemplars for retrieval is a promising approach, a framework which not only utilizes existing exemplars but collects new ones would improve the applicability of retrieval-based prompting approaches, to more diverse settings.

heading: Acknowledgements
text: This work was partly supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korean government (MSIT) [NO. 2021-0-01343, Artificial Intelligence Graduate School Program (Seoul National University)] and Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korean government (MSIT) [NO. 2022-0-00077, AI Technology Development for Commonsense Extraction, Reasoning, and Inference from Heterogeneous Data].

heading: A Appendix
text: A.1 Experiment details For LLM models, gpt-3.5-turbo-0613 and gpt-4-0613 are used through the OpenAI API. For the ablated RaA model, gpt-3.5-turbo-16k-0613 is used, since it
requires a larger input context to concatenate all subtask exemplars. For Mind2Web, we use gpt-3.5-turbo-16k-0613, following Zheng et al. (2024b). For all ϕ retrieval models, we use semantic similarity using text-embedding-ada-002 embeddings, over a retrieval corpus of task descriptions and metadata, following Zheng et al. (2024b).
A.2 Prompts A.2.1 CompWoB
CompWoB System Prompt
You are a large language model trained to navigate the web. To accomplish the task, use methods in the following Agent class to generate actions until you need the new state to proceed.
c l a s s Agent : d e f _ _ i n i t _ _ ( s e l f , a r g s ) :
. . .
# A c t i o n : t y p e a s t r i n g v i a t h e keyboard d e f t y p e ( s e l f , c h a r a c t e r s : s t r ) −> None :
. . .
# A c t i o n : c l i c k an HTML e l e m e n t wi th a v a l i d x p a t h d e f c l i c k _ x p a t h ( s e l f , x p a t h : s t r ) :
. . .
# A c t i o n s : p r e s s a key on t h e keyboard , i n c l u d i n g : # e n t e r , space , a r r o w l e f t , a r r o w r i g h t , # backspace , arrowup , arrowdown , command+a , # command+c , command+v d e f p r e s s ( s e l f , k e y _ t y p e : s t r ) −> None :
. . .
# A c t i o n : c l i c k an o p t i o n HTML e l e m e n t i n a l i s t w i th a # v a l i d x p a t h d e f c l i c k _ o p t i o n ( s e l f , x p a t h : s t r ) :
. . .
# A c t i o n : move mouse c u r s o r on an HTML e l e m e n t wi t h a # v a l i d x p a t h d e f movemouse ( s e l f , x p a t h : s t r ) :
. . .
RaD Subtask Decomposition
System: {system prompt}
Input: Here is a demonstration of interactions in the web environment, so you can get a sense of the environment: {retrieved exemplars for task}
Here is the task: {task}
Your goal is to decompose the task into a set of high-level subtasks. For now, do not consider their order, simply list them in the order that they are presented in the task. Start each subtask with [Subtask] and end each subtask with newline. End your generation with EOS.
RaD Subtask Planning
System: {system prompt}
Input: Here is a demonstration of interactions in the web environment, so you can get a sense of the environment: {retrieved exemplars for task}
Here are the subtasks, in no particular order: {subtasks}
Here is the precise task instruction: {task}
Now, generate the sequence of subtasks in the order that they should be executed to complete the task in the exact specification. Follow the format, 1. [Subtask]. 2. [Subtask]. 3. [Subtask]... End your generation with EOS.
Subtask Verification and Next Subtask
System: {system prompt}
Input: Here is the list of subtasks: {subtasks}
Here are the subtasks that have been completed so far: {completed subtasks}
Here are the actions that have been executed so far: {action history}
Write the next subtask that should be executed, in the format, [ID: n] where n is the id of the next subtask. End your generation with EOS.
RaA Action Generation
System: {system prompt}
Input: Subtask demonstration: {retrieved exemplars for subtask}
Now, here is the current task and state: {trajectory}
Here are the actions that have been executed so far: {action history}
Generate the action(s) for the following subtask: {subtask description}
A.2.2 Mind2Web Mind2Web System Prompt
You are a large language model trained to navigate the web. Output the next action and wait for the next observation. Here is the action space: 1. ‘CLICK [id]‘: Click on an HTML element with its id. 2. ‘TYPE [id] [value]‘: Type a string into the element with the id. 3. ‘SELECT [id] [value]‘: Select a value for an HTML element by its id.
RaD Subtask Decomposition
System: {system prompt}
Input: Now, you will be presented with a few trajectories of similar tasks. You can refer to them to understand what structure similar websites have, and what action trajectories are likely to be executable and successful. {retrieved exemplars for task}
Here is the task: {task}
Your goal is to decompose the task into a set of high-level subtasks. For now, do not consider their order, simply list them in the order that they are presented in the task. Start each subtask with [Subtask] and end each subtask with newline. Decompose into at most three subtasks. End your generation with EOS.
RaD Subtask Verification
System: {system prompt}
Input: Here are the subtasks that have not been completed yet: {remaining subtasks}
Trajectory: {trajectory}
Assuming that the previous action was executed successfully, write the subtask that was completed by the action. If no subtask was completed, write NONE. End your generation with EOS.
RaA Action Generation
System: {system prompt}
Input: Subtask demonstration: {retrieved exemplars for remaining subtasks}
Subtasks that have not been completed yet are: {remaining subtasks}
Trajectory: {trajectory}

heading: Future Work
text: While we did not consider retriever finetuning in this work, future research could consider an overall framework which finetunes the exemplar retriever in tandem with LLM agents. Another limitation of our method is that we leverage existing trajectories provided by the environments studied, as the corpora for exemplar retrieval. While the cost of collecting trajectories, as opposed to annotating prompt exemplars, is generally much lower, in some tasks human exemplars may be effective in bootstrapping the trajectory collection process. Finally, while leveraging existing exemplars for retrieval is a promising approach, a framework which not only utilizes existing exemplars but collects new ones would improve the applicability of retrieval-based prompting approaches, to more diverse settings