Agent Workflow Memory
Authors: Zora Zhiruo Wang
Jiayuan Mao
Daniel Fried
Graham Neubig
Venue: ICML 2025 poster (2025)
Keywords: agent
memory
web navigation
Primary area: applications
TL;DR: We propose Agent Workflow Memory that enable digital agents to induce and reuse increasingly complex workflows on the fly.

Abstract: Despite the potential of language model-based agents to solve real-world tasks such as web navigation, current methods still struggle with long-horizon tasks with complex action trajectories. In contrast, humans can flexibly solve complex tasks by learning reusable task workflows from past experiences and using them to guide future actions. To build agents that can similarly benefit from this process, we introduce Agent Workflow Memory (AWM), a method for inducing commonly reused routines, i.e., workflows, and selectively providing workflows to the agent to guide subsequent generations. AWM flexibly applies to both offline and online scenarios, where agents induce workflows from training examples beforehand or from test queries on the fly. We experiment on two major web navigation benchmarks — Mind2Web and WebArena — that collectively cover 1000+ tasks from 200+ domains across travel, shopping, and social media, among others. AWM substantially improves the baseline results by 24.6% and 51.1% relative success rate on Mind2Web and WebArena while reducing the number of steps taken to solve WebArena tasks successfully. Furthermore, online AWM robustly generalizes in cross-task, website, and domain evaluations, surpassing baselines from 8.9 to 14.0 absolute points as train-test task distribution gaps widen.

Agent Workflow Memory
Zora Zhiruo Wang' Jiayuan Mao? 2
Daniel Fried
Graham Neubig'
Abstract
Despite the potential of language model-based agents to solve real-world tasks such as web nav- igation, current methods still struggle with long-
Accumulative success rate (%)
40
Get driving time
from place A to
Display the route
Decide can drive from to a given time
Find tillonn hotel location, Eaen showe the shorest walking supermarkatt
30
horizon tasks with complex action trajectories. In contrast, humans can flexibly solve complex tasks by learning reusable task workflows from past ex- periences and using them to guide future actions. To build agents that can similarly benefit from this process, we introduce Agent Workflow Memory (AWM), method for inducing commonly reused routines, i.., workflows, and selectively provid- ing workflows to the agent to guide subsequent generations. AWM flexibly applies to both offline and online scenarios, where agents induce work- flows from training examples beforchand or from test queries the fly. We experiment two major web navigation benchmarks - Mind2Web and WebArena that collectively cover 1000+ tasks from 200+ domains across travel, shopping, and social media, among others. AWM M substan- tially improves the baseline results by 24.6% and 51.1% relative success rate on Mind2Web and WebArena while reducing the number of steps taken to solve WebArena tasks successfully. Fur- thermore, online AWM robustly generalizes in cross-task, website, and domain evaluations, sur- passing baselines from 8.9 to 14.0 absolute points as train-test task distribution gaps widen.' 1
Get the zip code a place
Get the 20 coondinates
a place
Find cafe
Gap widens 22.5 after 40 examples
10
near place
Find place by its name
0
AWM
Number of examples
baveline
Figure 1. AWM enables agents to continuously induce and apply workflows to improve performance, compared to stagnant hase- lines. We show results by AWM the WehArena map split as an example.
(Fu al., 2024; Murty et al., 2024) in-context learning (Zheng et al., 2024). This allows them to perform well on ac- tion sequences similar to those presented in these examples, but results in a lack of robustness to changes in task contexts or environments (Deng et al., 2023). Essentially, they fail to grasp the key to disentangling increasingly complex tasks to extract and learn reusable task workflows shared across similar tasks and environments (Yu al., 2023; Wang et al., 2024a). Moreover, as agents solve each task separately, they do not learn from past successes and failures, and are therefore unable to adapt over time (Yoran et al., 2024).
Motivated by how humans abstract common task routines from past experiences and apply such knowledge to guide future activities (Chi al., 1981: 2014), we propose agent workflow memory (AW M) ($2) to realize similar mech- anism in agents. AWM induces workflows from agent trajectories by extracting reusable routines, and then inte- grates these workflows into agent memory to guide future task-solving processes. Each workflow represents a goal with a common routine extracted from available action tra- jectories, which allows to effectively capture the most essential and reusable skills agents need acquire suc- cessfully solve increasingly complex tasks. As an example, Figure 1 shows workflows induced by AWM on the We- bArena map test split of the benchmark (Zhou et al., 2024), AWM starts with a basic set of built-in actions and solves new tasks in a streaming manner, continuously inducing workflows from the task at hand, e.g., learning to "find a
1. Introduction
Language model (LM) based agents are rapidly improving, and are now able to tackle digital tasks such as navigating the web (Zhou et al., 2024; Deng et al., 2023) or operating mobile apps (Rawles et al., 2023; 2024). Current agents mostly integrate a fixed set of given examples via training
'Carnegie Mellon University Massachussets Institute of Technology. Correspondence to: Zora Zhiruo Wang
Proceedings of the nnnd International Conference on Machine I Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s). http::/ggiii...ommmaaaaavagggggo http:///wwwwwwreaaoooooooootooom

---

Agent Workflow Memory
place by its name" from the first few examples. Moreover, AWM continues to build more complex workflows on top of new experiences and previously acquired workflows. For example, the "find a place by its name" workflow, once induced, effectively serves as a subgoal to build a more com- plex workflow "get the zip code of a place." Such continual learning mechanisms create a snowball effect to induce and apply increasingly complex workflows while expanding the agent memory, often yielding a substantial performance gap over a vanilla agent that does not adapt. This gap over the baseline rises as high as 22.5 points on WebArena after rolling over only tens of examples (as shown by Figure I).
2.1. Problem Statement
For the purpose of this paper, we consider agents with a lan- guage model backbone L and text-based memory M, where the base memory contains documentation of built-in actions such as CLICK and TYPE. 2 To solve task specified by a natural language (NL) instruction g. the agent acts in an environment defined by a transition function T. For each time step 4, the environment state 8; gives observation o,, which is then passed into the model to generate action a; via L(g, ox) - 0g. The action is executed in the envi- ronment and changes the state as T(s;, Q.) + 84+1. This observe-act loop iterates until the model predicts the stop action a, =STOP, or reaches a task termination condition, e.g., a maximum pre-determined number of steps.
AWM readily operates in both offline and online scenar- ios, where annotated examples are either available or non- existent. When high-quality annotated examples are avail- able for task, AWM operating in an offline fashion can extract reusable workflows from these canonical examples and integrate them into memory to assist test-time inference. Even if no annotated examples exist, AWM online can also run in an supervision- free setting, where it iteratively in- duces workflows from self- self-generated past predictions that are judged correct by an evaluator module.
Each completed task forms an experience e, which com- prises an NL instruction q and a trajectory of steps attempt- ing to solve the task, where each step p contains the agent observation o obtained from the current state, and action taken by the agent formulated as = (o, a). Our goal is to induce useful workflows W = {w} from the set of experiences ع = {e} constructed from past or collected examples, using an induction module via I(E) + W. We add induced workflows into the agent memory M as guidance for subsequent task-solving.
We evaluate AWM on two agent web navigation bench- marks ($3): WebArena, which provides rigorous execution- based evaluation (Zhou et al., 2024), and Mind2Web, which emphasizes broad tasks and domain coverage (Deng et al., 2023). On WebArena, AWM improves over the top pub- lished autonomous method (Drouin et al., 2024) by 51.1% relative success rate, and even outperforms methods using human expert written workflows (Sodhi al., 2023) by 7.9%. On Mind2Web, AWM effectively improves the cross- task results by 24.6% in relative step-wise success rate.
Next, we introduce the workflow representation design, workflow induction method, and agent memory update with workflows in varied setups.
2.2. Workflow Representation
Similar to an experience, workflow comprises two com- ponents: first, a textual description of the workflow d; and second, a series of steps to finish the workflow (p1. ".,...) as shown in Figure 2.
We further demonstrate the generalizability of AWM on both datasets. On WebArena, we create a cross-template subset where each example is instantiated from different task templates. AWM still consistently surpasses all baseline approaches, demonstrating its reliable cross-task workflow adaptability 1).. On Mind2Web, we evaluate AWM on the wrbss- and cross-domain test splits to examine its domain generality, where it scores 8.9-14.0 absolute points higher over baseline, and the margins become more substan- tial as the train-test distribution gap widens ($3.2). Both results show the superior generalization of AWM across tasks, websites, and domains.
Workflow Description To present workflows in format where agents can learn from them properly, it is impor- tant to describe the high-level goal of the series of actions. Therefore, we associate each workflow with an NL task description d, essentially a summary of the workflow's 's func- tion, by heuristically extracting from experience instructions or summarizing with an LM (see $2.3).
Workflow Trajectory The workflow trajectory contains series of steps (p1, P2, finish the process described in d. Each p consists of three parts, demonstrated in Pn in Figure 2, Step 3. (1) A description of the current envi- ronment state in NL, such as "Order {id} is shown"; (2) The reasoning process elaborated by the agent to decide which action to generate based on observations, such as "Order {id} is found, will now terminate the task."; and
2. Agent Workflow Memory
In this section, we first describe the web navigation task 1), then introduce the workflow representation ($2.2), and describe the mechanism of AWM as well as various
usage scenarios ($2.3).
Memory is usually implemented as a system prompt or auxil- iary information in the main prompt context.
2

---

Agent Workflow Memory
After the workflows W are induced, they are then integrated into the agent as auxiliary memory, M + W - M.., where M stands for the original agent memory, and Mi stands for the agent memory augmented with induced workflows.
ordered order HOL30?
Environment
Step 3.
Induce Workdlows
mucgnale rolo Hossar
Woeiflow Uiserpptandd df order with speiffedd Workdow lisuctory
action
Memory
When solving a given instruction q. the agent now produces a series of actions by L(g, Mw. o) = L(g, M + W, + a.
states
Agent
lene dool The niment cape shoss lattion] click) under link ad?)
observation
LM Backbone
lew dour Order (a0) shown Irasenn (kll is fond. I witl [action] stepi)
Next, we introduce AWM in use in two scenarios:
Offline Scenario AWM can operate in an offline scenario when additional canonical experiences are available, such as data annotated by humans synthesized by models. In this case, we perform workflow induction and utilization in two standalone processes. As shown in Figure 3 (left), AWM first takes in all training examples from a website by concatenating them into a single prompt, and feeds them to the LM to create set of workflows 'training' time; T ((Erran)) ) Woffime- Second, AWM incorporates all in- duced workflows into the agent memory at inference time to solve test instructions L(g, M + W Woffline of + afest Since the workflows are fully induced before test-time infer- ence, the agent uses the same workflow memory Woffine to solve each test.
Step
Oblain Actions Cannotaholgmmenet
<noodd click the "Oukkes" link to see all orders.
dlickt 1267 Wid of the bution
Trajectory Evaluation
order 0130 current page
scrolligt 2000
Query solved correctly?
The carrent page shows order 0130. send_ingg inng to inkin" Emura Lopez")
B
stupei
Figure 2. Illustration of our AWM pipeline: agent takes actions to solve given queries, induces workflows from successful ones, and integrates them into memory.
(3) an action represented as an executable program over the environment, i.e., stop () that realizes termination.
2.3. Inducing and Using Workflows
Online Scenario Extra canonical experiences are not al- ways available or easy to collect, especially those that cover the same domains and tasks as the test instructions. AWM also works in an online, supervision-free setting, where only test queries are needed. As in Figure 3 (right), agents with AWM M ontine process test queries in a streaming fashion, where the agents conduct the loop of induce, integrate, and utilize workflows after running inference for each test task.
At the core of AWM is an induction module I that induces a set workflows W from one or more past agent experiences 8 { 1m Each experience e = P*) contains an NL task instruction & and an action trajectory that consists of sequence of steps (observation and action) l = (pi,.... pf)) that were taken to solve q. The workflow induction module operates by taking in & and producing set of workflows, as (()) + W = [w} = (dd)
past
"Training" w/ extra examples
Infer test examples w/ workflows
workflows into memory
addling
Test examples
LM-based Workflow Induction To produce workflows that more accurately capture reusable trajectories across tasks, we propose an LM-based module that prompts the agent to extract common sub-routines from one or more input experiences.
stream
@induce Expply
indduce workflows
S apply workflows
emply workfluwe
workflow into memory
Different from task instructions that specify concrete, less- repetitive tasks, e.g., "Buy dry cat food on Amazon and deliver to my address", we deliberately prompt models to in- duce workflows at finer granularities, i.c., a sub-task "search for a product on Amazon" that frequently re-appears as part of multiple similar instructions. Meanwhile, instead of giving example-specifie values (e.g., "dry cat food"), we enhance workflow generality by abstracting out example- specific contexts, i.e., replacing "dry cat food"' with a more general name " "(product-name)" by specifying this in the workflow induction prompts. These workflows are seg- mented (based on double-line breaks in the model output) and stored separately in the workflow memory. Refer to $A for the exact model prompts, example workflows, and examination of quality.²
Figure 3. Illustration of AWM (left) and AWMoniiee (right).
Concretely, the agent starts with the default memory M; given the t-th test instruction q passed into the agent, the agent attempts to solve the task by generating an action trajectory joi.ph,... ...), which collectively forms an expe- rience et = (g", {p')). We adopt the LM--ased evalua- tion model of Pan et al. (2024) to output a binary label, Lenar(e") E {0, 1}. that judges if e successfully solves g' by prompting a neural model. If e' is predicted as success, i.e., we then transform into workflow(s) I(e") - {w'} and add {w"} into the agent memory M' fw"} + M which serves as the agent memory to process the /. + 1-th instruction. As depicted in Figure 3 (right), we continue this memory updattng process by iteratively predicting actions for and inducing workflows from streamed test instructions, until all tests are processed. We evaluate the success rate of
We also explore a rule-hased workflow induction method. See SB for more detailed experiments.
3

---

Agent Workflow Memory
predicted action trajectories {r²) for all tests.
success rate. According the breakdown on five websites, our AWM method substantially enhances the agent perfor- mance across all websites over the BrowserGym baseline, by 11.8-30.7 absolute points, indicating its general applica- bility across varied domains and tasks.
3. Experiments
In this section, we experiment on two major web naviga- tion benchmarks- - WebArena ($3.1) and Mind2Web ($3.2). For each benchmark, we first introduce the benchmark and uop-performing baseline methods, then present our AWM M approach and showcase its ability achieve superior task success and generalization across varied setups.
Beyond task success, we also evaluate the average number of steps the agent takes to solve a task, as shown in the right- most column in Table 1. AWM conducts about 2.0 fewer steps per example than the BrowserGym baseline. Further compared to the Autoeval (Pan et al., 2024) method, which necessitates additional evaluation and refinement steps to solve tasks correctly, our AWM approach uses 40.8 fewer steps on average. See $D for more efficiency discussions. Both comparisons show that AWM obtains high success rates while maintaining efficient trajectories.
For both benchmarks, we conduct AWM on a website ba- sis. In other words, we group examples by their associated websites, and respectively run AWM M on each group. This mechanism maintains a small collection of workflows that are nonetheless relevant to the test tasks.
3.1. WebArena
Accumulativve success (1))
40
WebArena (Zhou et al., 2024) provides 812 web navigation tasks on five websites that cover four common application domains:
30
e-commerce, social forum discussions, collabora- tive software development, and content management. Most importantly, WebArena supports rigorous evaluation on the functional correctness of agent trajectories.
Rapid
Stable
learning phase
inference phase
10
We adopt the current state-of-the-art method without human- annotated site- spec-sccciicc knowledge, BrowserGym (Drouin et al., 2024), which altered the agent default action space. We adopt the BrowserGym framework and its default action space, and represent webpages using accessibility trees, fol- lowing the environment representation in Zhou et al. (2024). Because BrowserGym inputs both webpage HTML and accessibility tree representations, to keep a fair compari- son with our method, we also run the BrowserGyy ver- sion with only accessibility tree webpage representations, denoted as BrowserGymaz--ee
20
40
60
80
100
Number of examples
Figure 4. AWM enables rapid learning from a small amount of data (about 40 queries), using WebArena map split as example.
3.1.2. EFFICIENT LEARNING FROM SMALL AMOUNTS OF DATA
To demonstrate the behavior of the AWM ontine method, we illustrate the cumulative success rate over the process of online evaluation, by evaluating the average success rate of the first k finished examples.
We also compare to the SteP method (Sodhi et al., 2023), which uses 14 human pert written workflows tailored to solving WebArena. Our method, in contrast, uses no human supervision and is not tailored to the WebArena setting.
As in Figure 4, the agent exhibits a fast learning curve in the beginning (between 0 examples), by acquiring the most essential workflows, which results in higher success rates. Afterward, agents learn more advanced workflows (Figure 1), while success rates gradually stabilize to the highest point in the early learning phase. This showcases AWM's efficient learning process, which substantially im- proves performance with merely tens of examples.
Following the baseline approaches, we use GPT-4o 4gpt-40-2024-055
with a temperature of 0.0 to en- sure mostly stable model outputs. Because WebArena only has test examples and no additional high-quality, domain- aligned examples exist, we only conduct AWM in the online setting as in $2.3.
1.3.
WORKFLOW GENERALIZATION
3. MAIN RESULTS
Some tasks in WebArena have highly overlapping canonical trajectories, due to the benchmark construction process that instantiates multiple examples from single underlying task template. AWM intuitively improves in-template success rate, that is, given one workflow induced from a successful example, would be theoretically easier to solve all other examples generated from the same task template.
As shown in Table I, our AWM achieves the best published results on WebArena, surpassing the BrowserGym base- line by 12.0 absolute points and 51.1% relative increase in overall success rate. Notably, AWM also outperforms SteP, which uses strong domain-specifer supervision from human- written workflows, by 7.6% relative increase in overall
4

---

Agent Workflow Memory
Table 1. Task success rate on WebArena using gpt-4, -4, and score breakdown on five website splits.
Method
Total SR
Shopping
CMS
Reddit
GitLab
Maps
# Steps
With human engineered workflows
*SteP (Sodhi et al., 2023)
33.0
37.0
24.0
59.0
32.0
30.0
Autonomous agent only
WebArena (Zhou et al., 2024)
14.9
14.0
11.0
6.0
15.0
16.0
AvtoEval (Pan et al., 2024)
20.2
25.5
18.1
25.4
28.6
31.9
46.7
BrowserGym (Drouin al., 2024)
23.5
BrowserGymaa tree
15.0
17.2
14.8
20.2
19.0
25.5
7.9
AWM (OURS)
35.5
30.8
29.1
50.9
31.8
43.3
5.9
To confirm that the benefits of AWM are not just from learn ing workflows that help only within a template, and investi- gate whether AWM M can obtain cross-template (sscross-taske generalization, we extract subset of WebArena examples sourcing from non-overlapping templates, by grouping amples by their templates and randomly choosing one ex- ample from each template group. We run AWM on this cross-template subset and examine if it achieves similar performance gains.
3.2. Mind2Web
Mind2Web (Deng et al., 2023) features web navigation in cross-task, website, and domain settings, stressing the gen- crality of agents on versatile operations and environments. Each task in Mind2Web has a fixed number of steps; at each step, the agent needs predict an action, which is evaluated by: (1) element accuracy: check if the correct page element is selected, (2) action F1 to check if the action taken on the element is correct, and aggregating (1) and (2) yields (3) step success rute which checks that both element and action selection are correct at the current step. Lastly, after completing every step in the given task, the last metric (4) task-level success rate measures if all intermediate steps are successfully conducted for this task, i.e., all steps for this task score 1 under metric (3).
As shown in Table 2, AWM still achieves the highest per- formance, overall and on each website split. These results demonstrate that AWM induced workflows effectively generalize across different tasks, i..., examples instantiated from different task templates.
Find a place by its name
Get the zip code of a place
Tock Objective: Show (location) the Trajectory:
Tack Objective: Tell zo code of (ocation)
Because Mind2Web provides a training set that covers part of the tested websites (the cross-task split), we explore both the offline setting that induces workflows from the training set and applies to test sets, and the online setting, where we stream workflow induction and inference on test queries ($2.3).
Action Trajectory,
a find the (location,, will search For "Jocation)" on Opendt MaEMa
Adoep the Buse tawe nadien Menes
Condireethigg bousd
687145
FRIC 45; (location)) diikk' 147))
The search results itss have provided miltiple
build Incrasinnlly
locacions... This location indudes the zip code.
send, meg. to www.urruuurru_.e. The zp code (stp-code)")
Figure 5. AWM builds increasingly complex workflows over time, by learning from past examples and carlier workflows.
Since we focus on LM-based agents that only take textual inputs, we compare AWM to two state-of-the-art methods, MindAct (Deng et al., 2023) and Synapse (Zheng et al., 2024). MindAct introduces webpage element filtering and multi-choice task format to ease observation processing, and Synapse changes the format to a trajectory style and aug- ments retrieved relevant examples. We integrate the element filtering adopted in both methods, and added workflows instead of retrieved examples in Synapse, to verify the supe- riority of reusable workflows over concrete examples. To fairly compare with all baseline methods, we run AWM with both gpt-33. 5-turbo and gpt-4 4 models with tem- perature 0.0. We use the same model for neural workflow induction and agent action generation.
Building increasingly complex workflows To more in- tuitively demonstrate AWM's cross-template generalization and ability to build increasingly complex workflows (as ex- emplified in Figure 1), we conduct case study to illustrate the workflow mechanism behind it.
As exemplified in Figure 5, the agent creates and learns the "Find a place by its name" workflow in the carly stage of the online process by summarizing past examples. Later, when encountering an example that further asks to obtain the zip code of the location, AWM agent learns adopt the first few steps find locations by following the existing workflow, and then conducts further steps obtain the zip code of the place found. Integrating these new steps upon the vanilla find location task, the agent successfully builds a more complex workflow, i.e., "get the zip code of a place". We further examine whether example ordering affects this gradual induction process in $E.
1.2. MAIN RESULTS
We first run with AWM offline using both GPT variants, and find that AWM consistently obtains the highest success rate in both step and task levels, leading to 4.0-8.9% rela-
5

---

Agent Workflow Memory
Table Task success rate on the cross-template subset of WebArena, as well as the result breakdown on each website split. We mark the number of examples for each website split under the name.
Shopping (51)
CMS
Reddit (24)
GitLab (45)
Maps (32)
Method
Total SR
(45)
With human engineered workflows
*SteP (Sodhi et al., 2023)
32.1
26.5
29.3
52.2
27.3
36.4
Autonomous agent only
AutoEval (Pan et al., 2024)
23.2
12.2
17.1
21.7
31.8
36.4
ProwserGymaz--fee AWM (OURS)
20.5
10.4
17.8
23.1
27.3
28.6
33.2
24.5
29.3
52.2
31.8
39.4
mented workflows may guide the agent to take certain ac- tions aligning to the workflows, which not always rel- evant to the particular environment state at hand. While following the workflows generally results in more success- ful task trajectories, agents still encounter challenges in identifying places to diverge from the workflow guidelines.
Table AWM offline results on Mind2Web cross-task dataset. Elem Ace and SR are short for element accuracy and success rate. We footnote the GPT variant used by each method, 3.5 and 4 stands for gpt- 5-turbo and gpt-4, respectively. We highlight the best result within the same model variant.
Method
Elem Ace
Action F1
Step SR
SR
MindActs.5
20.3
56.6
17.4
0.8
3.2.2. ONLINE AWM ENABLES GENERALIZATION
CogAgenta.s
18.6
Synapsea.s
34.0
30.6
2.4
Beyond the offline induction setting, we further explore the AWM in the online setting, similar to the WehArena experi- ment setup in $3.1, where no additional training examples are needed besides test queries. This naturally facilitates cross- website and cross cross-domain generalization, which we examine on the two other splits provided by the Mind2Web dataset: cross-website and cross-domain tests.
AWMs.0
39.0
52.8
34.6
2.8
MindActa
41.6
60.6
36.2
2.0
AWM,
50.6
57.3
45.1
4.8
tive and 0.4-2.8 absolute points increases in step-wise and task-wise success rates than the baselines Synapse with gpt-3. 5-turbo and MindAct with gpt-4. Decompos- ing the step success rate by element and action selection and accuracy, we notice the increases mainly come from more accurate element selection, as indicated by the 5.0-9.0 element accuracy increase in Table 3.
In addition the MindAct baseline, we additionally set bars with our AWMofftine setup, by randomly selecting work- flows induced from the training set as memory augmenta- tions. Specifically, for cross-website examples, we select workflows from the same domain; for the cross-domain setting, we randomly select workflows from all domains. We conduct AWM ontine by iteratively inducing, integrat- ing, and utilizing workflows over test inferences. We also explore oWMofftine+ online in &C.
A bstract sub-routines concrete experiences More specifically, compared to the Synapse (Zheng et al., 2024) method that retrieves the most relevant training examples, AWM achieves a +5.0 element accuracy and leads to a +4.0 increase in step success rate. While augmenting concrete, full examples may bias agents to select elements similar to those presented in the given examples. AWM introduces less bias element selection via its abstract representation of example-spccific contexts in workflows, and therefore enables higher step success rates.
As shown in Table 4, both AWM online and AWM offi surpass the MindAct baseline by large margin, resulting in 7.4-8.9, 3.6-3.8, and 14.0-16.9 absolute point improve- ments in step success rates, in cross-task, cross-website, and cross-domain scenarios.
In-domain, cross-task scenario When tested in-domain, AWMorline and AW AWMofRine perform comparably each other. When inspecting the model behaviors in detail, we notice the pros and cons of each method. AWM Monine in- duces workflows from model-predicted trajectories that are not always correct, thus can lead to incorrect workflows that degrade model performance. On the other hand, the training and test examples on some websites vary task distribu- tions (e.g., training examples cover how to buy items Amazon, test examples ask for job applications to Amazon careers.). AWM naturally resolves this train-test gap
Furthermore, AWM integrates frequently- sub-routines, which can be more flexibly and readily leveraged across test examples, compared to the full example trajectories used by Synapse, which are less likely appear multiple times. In general, our results indicate that the abstract, reusable nature of workflows contributes to the superiority of the AWM method.
Learn to diverge from workflow guidelines Despite more accurate element selection, AWM gets slightly lower action Fj scores than MindAct, possibly because the aug-
6

---

Agent Workflow Memory
Table Success rate Mind2Weh cross-task, cross wehsite, and cross-domain generalization test, using gpt-4 model. EA is short for element accuracy and AF, is short for action Fj.
Cross-Task
Cross-
Website
Cross-Domaii
Method
EA
AF1
Step SR
SR
EA
AF1
Step SR
SR
EA
AF1
Step SR
SR
MindAct*
41.6
60.6
36.2
35.8
51.1
30.1
21.6
52.8
26.4
2.0
AWMofflve
50.6
57.3
45,1
4.8
41.4
46.2
33.7
2.3
36.4
41.6
32.6
0.7
AWM online
oniine
50.0
56.4
43.6
4.0
42.1
45.1
33.9
1.6
40.9
46.3
35.5
1.7
because its operating process only involves test queries and environments, therefore yields workflows that are presum- ably more targeted toward the test distribution, which in turn, leads to higher success rates overall. Nonetheless, if distribution matthingg high-quality training examples are available, AWMoffinne could bring more benefit by alleviat- ing the gap issue, as the slightly higher cross- tasks scores of AW M offiine in Table 4.
steps whose action cannot be executed on the environment. We take these unique, validated experiences workflows. Find more detailed descriptions in $B.
Table 5. AWM success rate on WehArena using gpt-4, with rule- and in-based induction.
Method
Total SR
# Steps
AWM Mrude
35.6
6.3
Extending to unseen websites and domains When applied on unseen websites or domains, AWM online demonstrates greater generalization abilities, compared to AWMoffine. The performance margin of AWM (over AWMonfine) widens as the domain gaps between train- ing and testing data widen from different websites (e.g.. apple besthuy) to different domains (e.g., macys in shopping domain to reddit in social media domain). Be- cause AWM does not require nor rely on informa- tion from the training data, it is not affected by any do- main gaps. Nonetheless, as demonstrated by the substantial improvements of AWMoffiiee over the MindAct baseline, AWM still demonstrates that models can benefit from mechanistically similar workflows from the previously in- duced workflow repository.
AWM
35.5
5,9
WebArena Results As shown in Table 5, using rule- and b.s-d workflow induction performs comparably, with a small 0.1 gap in success rate; the L.M-baeed method appears more efficient and uses 0.4 fewer steps. Our manual analysis found workflows produced by the LM-based induction mod- ule Im are finer- -rainedd preventing agents from following unnecessary steps that sometimes appear in rule-induced workflows, hence making the task-solving process slightly more efficient.
Table 6. AWM results with different workflow induction methods on Mind2Weh cross-task dataset.
Method Elem Acc
Action F1
Step SR
SR
MindAct,
41.6
60.6
36.2
2.0
4. Exploring Optimal Workflow Representations
AW Muee
49.5
57.0
43.4
2.0
AWM4.6m 50.6
57.3
45.1
4.8
In this section, we experiment with other possible alterna- tives to better represent the workflows. Specifically, we ablate workflows in sub-routine, abstract formats ($4.1), explore workflows in descriptive texts ($4.2), and lastly, be- yond the default workflows that describe environment state in NL, we compare strengthened observations with website HTML within workflow steps ($4.3).
Mind2Web Results In Table 6, compared to AWMrudes AWM improves by 2.8 margin. While augmenting concrete, full examples may bias agents to select elements similar to those presented in the given examples, AWM in introduces less bias on element selection via its abstract representation of example-specific contexts in workflows.
Further, AWM in uses frequently-used sub-routines, which can be more flexibly and readily utilized across test exam- ples, compared to the full example trajectories induced by AWM rule, which are less likely to appear multiple times. In general, our results indicate that the abstract, reusable nature of workflows contributes to the efficacy of AWM M Im method.
4.1. How much does the sub-routine, abstract format contribute?
In this section, we compare our abstract, sub-routine- induction method using LMs a rule-based method without context and sub-routine abstraction.
Specifically, our rule-based induction Trute first extracts the action sequence (e.g., CLICK- CLICK - TYPE) of each experience and deduplicates experiences by their action sequence. In each unique experience, we then remove the
4.2. Workflows in Deseriptive Texts
AWM represents workflow steps in a program format. In this section, we compare with a textual format for work-

---

Agent Workflow Memory
flows, to understand whether text or code serves better format for agent memory. More concretely, we prompt t-tuuro to verbalize the action trajectory in the workflows induced in earlier experiments. For example, from an action CLICK ((submi t-idd), its verbalized NL representation reads similar to "CLICK the submit but- ton". We use the same textual observation and thoughts from code actions as observation and thoughts in these text actions.
5. Related Work
Web Agent Benchmarks The first modern and widely used web agent benchmark is Shi et al. (2017)'s MiniWob, which evaluates across various scenarios such as flight book- ing. (Liu et al., 2018) then created MiniWob++ with extra challenges. More recently, WebShop (Yao et al., 2022) fea- tures a simulated e-commerce website and crowd-sourced text instructions. WebArena (Zhou et al., 2024) integrates four more websites and enables realistic execution based evaluations, and VisualWebArena (Koh et al., 2024) extends with tasks that necessitate visual inputs. Mind2Web (Deng et al., 2023) proposes versatile tasks and stresses agent gen- eralization across websites and domains. We use WebArena and Mind2 to evaluate our method's task success and generality.
Table 7. Mind2Wch cross-task results with AWM using code and
text workflows.
Method
Elem Ace
Action Fi
Step SR
SR
MindAct
41.6
60.6
36.2
2.0
AWM
50.6
57.3
45.1
4.8
AWM 51.2
57.4
45,4
3.6
Enhancing Agents for Complex Tasks Many works im- prove agents by modifying their action space, such as con- straining its action search space (Liu et al., 2018), enabling LM self-feedback to refine predicted actions (Sun al., 2023), incorporating human-designed actions to certain tasks (Sodhi et al., 2023; Sarch et al., 2024). Other works explore ways to augment agent memory, such as adding example demonstrations in context (Haluptzok et al., 2023; Zheng et al., 2024; Fu al., 2024). However, high-quality examples are not always available or easy to collect. Our AWM can flexibly operate even when auxiliary examples are non-existent and only test queries are available.
From the results in Table 7, AWM achieves slightly higher element selection accuracy and step success rate, by 0.6 and 0.3 points, respectively, yet degrades 1.2 task suc- cess rate. Overall, we do not find substantial performance variance between workflows represented in text and code formats, indicating that both forms can be effective. See $F for more studies on executable action workflows.
4.3. Environment Abstraction in Workflows
AWM describes intermediate webpage states using NL, yet showing concrete states may be helpful to better ground agents on the environment. Since webpage's 's full HTML. can be overly long, we filter the webpage representation using the relevance predictor of Deng et al. (2023), and augment each workflow step with this shortened HTML. that only has elements predicted as relevant. We run gpt-3. 5 turbrbo with only descriptions, only HTML, and both types of content.
Learning Common Procedures from Experiences Some works use full examples (Zheng et al., 2024) as con- text for an agent, yet they entangle with example-speciffe contexts and face challenges extrapolating to other tasks or domains (Majumder et al., 2023). Many works propose to extract frequently reused sub-routines from experiences with rule-based (Ellis et al., 2023; Bowers et al., 2023; Grand et al., 2023) or LM-based methods (Cai et al., 2023; Wang et al., 2024c;a) methods, and use them as auxiliary skills to ease future task-solving (Oh et al., 2017: Liang et al., 2023; Yu et al., 2023; Mao al., 2023). We explored both nule- and LM-dased methods to induce reusable workflows. and use them flexibly context guidance that are free of environment grounding issues.
Table Mind2Web results using GPT-3.5-turbo with different en- vironment representations.
Desc.
HTML
Elem Ace
Act F
Step SR
SR
x
39.0
52.8
34.6
2.8
/
38.1
54.0
33.8
2.8
37.1
51.3
32.9
2.0
As shown in Table 8, NL description of states is more useful than HTML, as replacing NL with HTML leads to a slight 0.8 drop in step success rate. Interestingly, using both NI. and filtered HTML leads worse results. We conjecture the reason be two-fold. First, adding NL and HTML sub- stantially increases the context length, thus making it harder for models to handle things correctly. Second, the filtered HTML has substantial number of irrelevant items (miss- ing all correct elements 47% of the time) thus potentially contradicting NL. descriptions and impairing agent abilities.
6. Conclusion
We propose agent workflow memory that induces, augments, and uses workflows, offline from available examples or purely online at inference time. We evaluate AWM on WebArena and Mind2Web, and achieve 24.6% and 51.1% relative increases in task success rate. AWM also demon- strates its superior generalization abilities across tasks, web- sites, and domains. We hope AWM sheds insight on and boosts advances in dynamic memory building and agent adaptations on varied digital tasks.
8

---

Agent Workflow Memory
Acknowledgments
learning. Philosophical Transactions of the Royal Society A, (..18255):202200550 2023.
We thank Frank Xu, Jiayi Pan, Vijay Viswanathan, Chenglei Si, and Jason Wu for their helpful discussions during the early stage of this project. We would like thank members of NeuLab and DFried Lab at Carnegie Mellon University for their valuable feedback and comments the paper. Zora Zhiruo Wang is supported by the CMU Presidential Fellowship and Fujitsu Research.
Fu, Y. Kim, D.-K., Kim, J., Sohn, S., Logeswaran, L., Bae, K., and Lee, H. Autoguide: Automated generation and selection of state-aware guidelines for large language model agents. arXiv preprint arXiv:2403.089 2024.
Grand, G., Wong, L.,, Bowers, M., Olausson, T. X., Liu, M., Tenenbaum, J. B., and Andreas, J. Lilo: Learning interpretable libraries by compressing and documenting code. arXiv preprint arXiv:2310. 19791, 2023.
Impact Statement
This paper presents work whose goal is advance large language model-supported digital agents. While utilizing digital agents in practice can facilitate users in completing computer-using tasks, these agents may not perform per- fectly accurately, and ensuring safety and privacy through- out the usage is still an open problem in the field.
Haluptzok, P., Bowers, M., and Kalai, A. Language models can teach themselves program better. In The Eleventh International Conference Learning Repre- sentations, 2023. URL http://ppeanvvvw..efffommm SaRj2ka1XZ3.
Koh, J. Y., Lo, R., Jang, L., Duvvur, V., Lim, M. C., Huang, P.-Y., Neubig, G., Zhou, S., Salakhutdinov, R., and Fried,
References
D. Visualwebarena: Evaluating multimodal agents on realistic visual web tasks. In ICLR 2024 Workshop
Bowers, M., Olausson, T. X., Wong, L., Grand, G., Tenen- baum, B., Ellis, K., and Solar-Lezama, A. Top-down synthesis for library learning. Proc. ACM Program.
Large Language Model (LLM) Agents, 2024. URL https: wwtp:////ooommmmmn//P..o.
Lang., ((PPLL) jan 2023. doi: 10.1145/3571234, URL http://doi.or/1101115553333
Liang. J., Huang, W., Xia, F., Xu, P., Hausman, K., Ichter, B., Florence, P., and Zeng, A. Code as policies: Language model programs for embodied control. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 9493 9500. IEEE, 2023.
Cai, T., Wang, X., Ma, T., Chen, X., and Zhou, D. Large language models as tool makers. arXiv preprint 2rXiv:2305. 17/26, 2023. URL https:/arriiv.rr//dff22000. 17126.
Liu, E. Z., Guu, K., Pasupat, P., and Liang, P. Reinforcement learning on web interfaces using workflow-guud guided explo- ration. In International Conference on Learning Repre- sentations, 2018. URL http:///peenevve..eettoummmaa ryTp3f-0-.
Chi, M. T., Feltovich, P. J., and Glaser, R. Categorization and representation physics problems by experts and novices. Cognitive science, 5(2):121-152, 1981.
Chi, M. T., Glaser, R., and Farr, M. The nature of expertise. Psychology Press, 2014.
Majumder, B. P., Mishra, B. D., Jansen, P., Tafjord, O., Tandon, N., Zhang, L., Callison-Burch, C., and Clark, P. Clin: A continually learning language agent for rapid task adaptation and generalization. arXiv preprint arXiv:2310.101 10134, 2023.
Deng, X.. Gu, Y., Zheng, B., Chen, S., Stevens, S., Wang, B., Sun, H., and Su, Y. Mind2web: Towards gener- alist agent for the web. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023. URL https://openreieewneell forum"id=kiyybO3wqqw.
Mao, J., Lozano-Pérez, T., Tenenbaum, J. B., and Kael- bling, I.. P. Learning reusable manipulation strategies. In Conference on Robot Learning, pp. 1467-1483. PMLR, 2023.
Drouin, A., Gasse, M., Caccia, M., Laradji, I. H., Del Verme, M., Marty, T., Boisvert, I.,, Thakkar, M., Cappart, Q., Vazquez, D., et al. Workarena: How capable web agents at solving common knowledge work tasks? arXiv preprint arXiv:2403.07711 2024.
Murty, S., Manning, C., Shaw, P., Joshi, M., and Lee, K. Bagel: Bootstrapping agents by guiding exploration with language. arXiv preprint ar arXiv:2403.08)
2024.
Ellis, K., Wong, L.,
Nye, M., Sable-Meyer, M., Cary, L., Anaya Pozo, L., Hewitt, L., Solar-Lezama, A., and Tenen- baum, J. B. Dreamcoder: growing generalizable, inter- pretable knowledge with wake-sleep bayesian program
Oh, J., Singh, S., Lee, H., and Kohli, P. Zero-shot task gen- eralization with multi-task deep reinforcement learning. International Conference on Machine Learning, pp. PMLR, 2017.

---

Agent Workflow Memory
Pan, J., Zhang, Y., Tomlin, N., Zhou, Y., Levine, S., and Suhr, A. Autonomous evaluation and refinement of digital agents. arXiv preprint arXiv:2404.066. 06474, 2024.
Yao, S., Chen, H., Yang, J., and Narasimhan, K. Webshop: Towards scalable real-world web inter- action with grounded language agents. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Infor- mation Processing Systems, volume 35, pp. 20744- 20757. Curran Associates, Inc., 2022. URL htips: http://seeerrppsppppppppeeieeeee 82ad1 3ec0 19fc44401cb91 114111881411111111 Papeaaa Confeeeeee......ceeee. pdf.
Rawles, Li, A., Rodriguez, D., Riva, O., and Lillicrap, T. P. Androidinthewild: A large-scale dataset for android device control. In Thirty-seventh Conference on Neu- ral Information Processing Systems Datasets and Bench- marks Track, 2023. URL https://ppeneviee.neffoomm id=j4b315kOil.
Rawles, C.,, Clinckemaillie, S., Chang, Y., Waltz, J., Lau, G., Fair, M., Li, A., Bishop, W., Li, W., Campbell- Ajala, F., et al. Androidworld: dynamic benchmark- ing environment for autonomous agents, arXiv preprint arXiv:2405. 14573, 2024.
Yoran, O., Amouyal, S. J., Malaviya, C., Bogin, B., Press, O., and Berant, Assistantbench: Can web agents solve realistic and time-consuming tasks? arXiv preprint 2024.
Yu, W. Gileadi, N., Fu, C., Kirmani, S., Lee, K.-H., Are- nas, M. G., Chiang, H.-T. L.,, Erez, T., Hasenelever, L., Humplik, J., et al. Language rewards for robotic skill synthesis. arXiv preprint arXiv:2306.0880 086477 2023.
Sarch, G., Jang, L., Tarr, M. J.. Cohen, W. W., Marino, K., and Fragkiadaki, K. Ical: Continual learning of multi- modal agents by transforming trajectories into actionable insights. arXiv preprint arXiv:2406. 11te224466.1496, 2024.
Zheng, L., Wang. R., Wang. X., and An, B. Synapse: Trajectory sngjectory-as-eeenee exemplllll prompting with memory for com- puter control. In The Twelfth International Confer- ence on Learning Representations, 2024. URL https:
Shi, T., Karpathy, A., Fan, L.,, Hernandez, J., and Liang, P. World of bits: An open-domain platform for web-based agents. In Precup, D. and Teh, Y. W. (eds.), Proceed- ings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learn- ing Research, pp. 3135 3344.444 PMLR, 06-11 Aug 2017. URL https://proceedings. wwtppp//////////////
Zhou, S., Xu, F., Zhu, H., Zhou, X., Lo, R., Sridhar, A., Cheng, X., Ou, T., Bisk, Y., Fried, D., Alon, U., and Neubig, G. Webarena: A realistic web environment for building autonomous agents. In The Twelfth International Conference on Learning Representations, 2024. URL http://ownneeeeeeeee//omm/iiiaio
Sodhi, P., Branavan, S., and McDonald, R. Heap: Hierar- chical policies for web actions using lims. arXiv preprint arXiv:2310.03770, 2023.
Sun, H., Zhuang, Y., Kong, L., Dai, B., and Zhang, C. Adaplanner: Adaptive planning from feedback with lan- guage models. In Thirty-sevnnth Conference Neu- ral Information Processing Systems, 2023. URL https: Hiopeneeview.eetfforrm Kid=nnggKKeeettt
Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, I.,, and Anandkumar, A. Voyager: An open-ended embodied agent with large language mod- els. Transactions on Machine Learning Research, 2024a. ISSN 2835-8856. URL https://peemevioo.n//ffommm ehfRiFOR3a.
Wang, Z., Cheng, Z., Zhu, H., Fried, D., and Neubig, G. What are tools anyway? survey from the language model perspective. In First Conference on Language Modeling, 2024b. URL. http:://ppeemereieeeeee//ommnnn XhIB90iBSE
Wang, Z., Neubig, G., and Fried, D. TroVE: Inducing veri- fiable and efficient toolboxes for solving programmatic tasks. In Forty-first International Conference on Machine Learning. 2024c. URL https://oennevio..ee/fommm/ DCNCwaMIjI.
10

---

Agent Workflow Memory
A. LM-Based Workflow Induction
As introduced in $2.3, one realization our workflow induction module to prompt LMs to generate abstract, sub-routine workflows from the given examples, i.e, experience. this section, we provide the detailed model prompt, exemplar workflows induced by models, and quality examination on these workflows.
A.1. Model Prompt
We provide the exact prompt inputted the model for WebArena and Mind2Web experiments below. Experiments on both datasets use the same prompt.
Given list of web navigation tasks, your task to extract the common workflows.
Each given task contains natural language instruction, and series of actions solve the task. You need to find the workflow.
repetitive subset actions across multiple tasks, and extract each of them
Each workflow should be commonly reused sub-routine of the tasks. Do not generate similar overlapping workflows. Each workflow should have at least two steps. Represent the non-fixed elements (input text, button strings) with descriptive variable names as shown in the example.
A.2. Example Workflows
We present several exemplar workflows induced on WebArena and Mind2Web, to give a more concrete impression of workflows.
WebArena Workflows We show one example workflow on each website involved in WebArena.
## shopping: Browse Products in a Specific Category
To browse products in specific category, need navigate the relevant main category. I will start by hovering over the main category menu item to reveal the subcategories. hover (' main.category-id
To browse products in the specific subcategory, I need to click on the subcategory link.
el icck (' subcategory.idd
## shopping admin: Edit and Save Changes
This workflow is used to edit specific fields and save changes.
To edit a specific field, need to locate the field and update its value.
clear (' field )
('
new_value"s
Next, I need to save the changes by clicking the "Save" button. click (' save )
## reddit: Navigate to forum section and select specific forum To navigate to a specific forum, need to click on the "Forums" section.
click (' 42')
Now, need to click on the specific forum link based on the forum name provided.
click (' <forumllink.id:
## gitlab: Navigation to Repository and Contributors Section
This workflow involves searching for repository and navigating to its contributors find detailed contribution data.
First, search for the specific repository to gather information.
fill (' (3''0', {RepositoryName)" press (' 130', 'Enter')
Navigate to the "Contributors" section to view contribution details.
click (' 311') # "Contributors" link
Obtain and report the required contributor details.
send msg to user (' (Contributee rituttrrrrrrraalss)))))

---

Agent Workflow Memory
## map: Calculate Travel Time and Distance
To calculate travel time and distance between two locations, will use the directions feature. will fill in the respective fields and select the mode of transportation.
fill ('1'' "FROM LOCATION')
fi TO.LOCATTONN)
select.option (' 166'' "MODELOF TRANSPORTATION
click ('171') I will use these details to provide the user with accurate travel time and distance information.
send mag to user (' The dianance between FROM LOCATTON and TO LOCATION is DISTANCE and the estimated travel time is TIME.')
Mind2Web Workflows We present one example workflow in each data domain in Mind2Web.
# travel: enter flight locations Given that you are on the flight booking page, this workflow enters the departure and destination city/airport for your flight.
[link] From Departure Airport or City Your Origin - > CLICK
[textbox] Origin City Airport - > TYPE: (your-origin-city)
[link] (best-popup-optionn)
CLICK [link] To Destination Airport or City Your Destination > CLICK
[textbox] Destination City or Airport > TYPE: (your-destination-ity
[link] (best-popup-option) > CLICK
# shopping: search_an search_and_sov anr_sort
Given that you are on the Amazon search results page, this workflow searches for a product and sorts the results.
[textbox] Search Amazon TYPE: fsearch-term]
[button] Go > CLICK
[span] Sort by: CLICK
[option] {sort-option] - > CLICK
# entertainment: search and select Given that you are the IMDb homepage, this workflow searches for term and selects the best match.
[textbox] Search IMDb TYPE: {scarch-term}
[button] Submit Search > CLICK
[button] [best-match]- - CLICK
A.3. Workflow Quality Analysis
To provide intermediate information beyond the end-to-end task success, we propose several metrics to verify the quality of the model-induced workflows. (1) Number of workflows: The number of workflows augmented to the memory, fewer workflows better, whereas agents rely fewer workflows achieve satisfactory performance. (2) Coverage: How many steps in the action trajectory are covered by the workflows, higher coverage presumably signals the general applicability of the concerned workflow. (3) Function overlap: How much functionality overlap exists between workflows, we measure this by counting the number of overlapping sub-trajectories (< 2 steps) between each workflow pair for the same website. Less overlap indicates more maximized workflow management. (4) Utility rate: How often are workflows used by test examples.
evaluate the workflows on WebArena test examples and Mind2Web cross-task test examples. We do not evaluate coverage on WebArena since it requires canonical trajectories, yet which are not available for WebArena. For Mind2Web. do not evaluate cross-weths boss-website and cross-domain test examples since workflows induced from training examples do not have domain overlapping with these test examples, thus less applicable to them.
As shown in Table 9, neural-based induction produces 7.3-7.4 workflows per example, which is efficient and do not add too much content to the memory. On WebArena, the induced workflows are used by 0.94 of the test examples, indicating its wide upplicability among varied tasks. Further, only 0.08 of the steps between workflows overlap, demonstrating the efficiency of workflows solving respective tasks. Workflows on Mind2Weh, although used similarly frequently as indicated by the high
12

---

Agent Workflow Memory
Table 9. Quality evaluation of model-induced workflows Mind2Weh dataset.
Metric
Workflows
Coverage
Function Overlap
Utility Rate
WebArena
7.4
0.08
0,94
Mind2Web
7.3
0.40
0.20
0.91
0.91 utility rate, have slightly more functional overlap, and only achieve 0.40 coverage over test examples. However, as the training examples used to induce workflows have substantial task distribution variances with the cross-task test examples, this relatively low coverage is reasonable.
B. Rule-Based Workflow Induction
Beyond LM-based workflow induction, we also explored a rule-based workflow induction method. Our rule-based workflow induction module consists of two steps: (i) experience deduplication, and (2) invalid action filtering.
For deduplication, we extract the action sequence of the experience, c.g., extracting CLICK + CLICK + TYPE from the trajectory CLICK (' ) - CLICK (' 30') + TYPE (' 44', "cat"). We group experiences by their action sequence and randomly select n (n 1 by default) experiences from each group. Specifically on WebArena, where the task template for each experience is available. We conduct another round of deduplication by groupngg experiences by their task template, and randomly selecting n (n by default) experiences from each group. This process yields diverse experiences from the given set of experiences.
Next, for each unique experience, we remove the invalid steps in its action trajectory. Invalid actions means actions that cannot be successfully executed on the environment, because the input arguments do not meet the requirement of the action function. Specifically, we have one rule determining invalid actions for CLICK and TYPE, that requires the first argument to be string-formatte
integer (which refers the id of an element in the environment). We remove CLICK and TYPE steps if they do not meet this requirement. For example,
experience with trajectory CLICK (12) - CLICK (' 12') + CLICK (1 30') + TYPE (44, " "cat") + TYPE (' 44', "cat") will yield CLICK (' 12') - CLICK (' 30') TYPE (' 44', "cat"). We conduct this invalid action filtering for each unique experience, and take the resulting experiences as rule-based workflows.
-
C. Integrating AWM Offline and Online
We compared AWM Moffinee and AWMonline in $3.2, that adopts workflows induced separately from training or on-the-fly during testing, respectively. In this section, explore integration of both sets of workflows, A that injects relevant training workflows to warm start task-solving, but also aggregates increasingly more online-induced workflows to better adapt to test distributions.
Table 10. Success rate on Mind2Web cross-task, cross-website, and cross-domain generalization test, using gpt -4 model. EA is short for element accuracy and AF, is short for action F,.
Cross- Cross-Task
Cross-
Website
Method
EA
AF:
Step SR
SR
EA
AF:
Step SR
SR
EA
AF1
Step SR
SR
MindAct"
41.6
60.6
36.2
2.0
35.8
51.1
30.1
2.0
21.6
52.8
18.6
1.0
AWM offisne
50.6
57.3
45.1
4.8
41.4
46.2
33.7
2.3
36.4
41.6
32.6
0.7
AWM online
50.0
56.4
43.6
4.0
42.1
45.1
33.9
1.6
40.9
46.3
35.5
1.7
AWM on
50.0
57.0
44.5
1.6
41.8
45.5
33.3
1.1
39.3
44.3
34.1
1.5
From Table 10, AWM on scores between AWMogiane and AWMonliine across three test splits. Rather than an additive
effect, workflows induced offline and online are not fully compatible with each other, particularly, the offline workflows seem to impair the generative quality and utility efficacy of online workflows, therefore resulting in medium results overall.
D. Efficiency Discussion
In addition to task success rate, the efficiency and scalability of agents are crucial too. In addition to reporting the number of steps in $3, we provide a more detailed analysis of the computation cost for each of the modules involved in AWM.
13

---

Agent Workflow Memory
Besides the action generation step, our AWM approach adds two other steps trajectory evaluation and workflow induction. We calculate the average computation of all three steps by the number of input, output, total tokens per step, the average number of times that the step occurs per task, and the total number of tokens used average for task. As shown in Table 11, the trajectory evaluation step and workflow induction step only take 4.0% and 6.8% of the compute of the original action generation step. Compared to the baseline method (using action generation step only), our AWM approach only adds 10.8% computation overhead, but brings a 51.5% accuracy increase in WebArena tasks, demonstrating the cost-effectiveness of our AWM approach.
Table I1. Computation cost breakdown for action generation, trajectory evaluation, and workflow induction modules in AWM
Step
# Input Tokens
Output Tokens
# +Per Step Tokens
# +ccurance
# Total Tokens
Action generation
5,663
52.0
5,715
5.9
33,718.5
Trajectory evaluation
306.8
82.8
389.6
5.9
2298.6
Workflow induction
306.8
328.7
635.5
2.1
1344.6
E. AWM Sensitivity to Example Ordering
The process of AWM gradually learning increasingly complex workflows may raise the hypothesis that, streaming the examples in an easy-to-hard order may facilitate this gradual learning process and optimize AWM performance. Therefore, we test the effect of example ordering in AWM experiments. We conduct this analysis on the Mind2Web dataset (cross-task split); because WebArena examples need to be kept in the original order released to maintain the validity of the browser environment (Zhou et al., 2024), we could not change the orders arbitrarily to examine this effect. We run the AWM online approach to focus on test example ordering without being affected by other training examples during the offline process. Specifically, we compare the (1) original ordering, (2) random shuffling as a comparison baseline, (3) easy-to-hard ordering, and (4) hard-to-easy ordering. For (3) and (4), we measure the examples' difficulty by the number of steps in the ground-truth trajectory, i.e., the more steps, the harder the example. The results are shown in Table 12.
Table 12. Success rate on Mind2Web (cross-task) with AWM onlime different example ordering.
Method
Element Ace.
Action F;
Step SR
SR
MindAct
41.6
60.6
36.2
2.8
Original
50.6
57.3
45.1
4.8
Random shuffle
49.4
57.9
45.9
Easy-to-hard
49.8
57.8
45.7
Hard-to-easy
48.5
59.0
45.6
4.2
First, AWM in all example orderings still substantially outperforms the MindAct baseline. Moreover, the ordering of examples does not significantly affect the performance of our AWM approach, where all four example ordering achieves similar step success rates. Coupled with careful analysis of the derived workflows, we found that our design of the "sub-task level" workflow contributes AWM robustness example ordering - regardless of the complexity of the task, method can induce usable workflow. Nonetheless, because each website in the Mind2Web dataset only has less than 20 examples, A'M's robustness to example ordering in relatively small numbers of examples (as shown by the experiments above) may not fully extrapolate larger datasets.
F. Exploring Workflow Utilization in Context and in Action
Besides integrating workflows as agent memory, we also explore workflows expanding the agent action space, denoted as AWMAS. We leverage the programmatic nature of workflows and wrap each workflow into a high-level function, similar to shortcut tool the agent can call to perform pre-determined series of actions (Wang et al., 2024b). Formally, an agent is initially equipped with default, primitive actions P (e.g., click, type), and AWMAs adds the induced workflow actions W (e.g., find place, get place z zipcode) to its action space.
The agent can call a primitive or workflow action at each step. When a primitive action is called, the agent immediately takes that action. When the agent calls a workflow action, it will trigger the sequence of pre-determined steps in the workflow. For example, calling the login (username, password) workflow action results in sequentially executing
14

---

Agent Workflow Memory
click (boxl-id) + type (boxl-id, username) + click (box2-id) - type (box2-id, password)
+cick (submit-id) The workflow action is completed when all intermediate primitive actions are finished.
In Table 13, expanding the agent action space with work- flows (AWMAs) slightly improves the step success rate by 1.3 points, and gets the same overall success rate, 3.2, the base memory-augmented AWM. We analyzed agent predictions and found they call workflow actions in merely 18.5% of the tasks, suggesting resistance of cur- rent agents
Table 13. Mind2Web results with AWMAs variant that alters the action space besides memory augmentation. All methods use gpt-4.
Method
Elem Acc
Action Fi
Step SR
SR
MindAct
41.6
60.6
36.2
2.0
AWM
50.6
57.3
45.1
4.8
newly-added actions. Overall, expand- ing actions with workflows seems reinforce workflows in memory, and brings small extra gains as auxiliary actions.
AWMAS
51.8
56.7
46,4
3.6
However, workflow actions do not always lead to task success. A representative example is shown in Figure 6. When booking flights, users often input a city name such as "New York," yet the system often pops up some nearby airports to support next-step search. While one can induce book flight workflow that enters all required data via pre-determined action sequence, the action choose pop-up airports is executed without seeing the intermediate states with available pop-up options, and is not flexible enough to do so. More advanced techniques such as granting real-time state access or dynamic execution loops can be promising to solve this issue, and we encourage future work to leverage the AWM framework to explore these.
Fiy Estatcs
C Rondkii
My Migi
2
Flight stibus
Check-in
My trip
Packages
Hotal
Car
Cruise
Rexteedates
Boocweth
Ptttbburhh
Jestiarion
Pribbugh
Now Yory
Environmentt
Dates
Traoelers
Dates
oucozenco
Oct 18 Oct 20
1AdUk
Oct Oct 20
B
Now US (nporpp
US (UGR)
Econemy
Exenomy
Now York, UB (FFK)
Pind Rights
Adiaredd search
(Oer MoriBoaauuuleee appeed)
(our oby sigitiaa
Albumy, NO (AUB]
Find Finiyourtinn trovel Gredits
Depend on the pop up options
Action
click(120) # of textbox under "o typc(120, "New York") # enter location
select('New York, NY, US (JFK))
Figure 6. An example of dynamic environment changes that challenge workflow action utilization.
15


Reviews:
[{"review_id": "hH2AMWqyEf", "reviewer": "Reviewer_aBUC", "summary": "The authors introduce Agent Workflow Memory (AWM), a method that enables agents to learn and reuse task workflows, similar to how humans leverage past experiences to solve complex tasks. AWM identifies commonly used routines and selectively provides them to guide future actions in both offline and online settings. Applied to the Mind2Web and WebArena benchmarks, AWM improves baseline success rates by 24.6% on Mind2Web and 51.1% on WebArena while reducing steps needed for successful navigation. Also, online AWM generalizes robustly across tasks, websites, and domains, outperforming baselines by 8.9 to 14.0 absolute points.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for recognizing the effectiveness of our AWM approach!\n\n**AWM’s robustness to varied offline task selections**\n> We agree that offline task selection is crucial for AWM to achieve better performance, where higher distribution overlap between offline/training tasks and online/testing tasks improves agent performance. As we directly adopt the Mind2Web dataset in our experiments on Mind2Web (to fairly compare with existing methods), its three test sets (as introduced in sec 3.2 beginning) already introduce mismatches in tasks, websites, and domains to those covered in the training examples. As shown in Table 4, our AWM still outperforms baselines on all three test sets, demonstrating its robustness when the train-test task distribution varies to different extents.\n\n**Extend memory beyond subroutines**\n> We agree with the reviewer that memory can take various forms. Our work aims to take a first step in learning one of the most crucial memory types -- reusable, procedural workflows -- that boost agent success and efficiency. It would be a promising direction to extend this adaptive AWM method to other types of memories, such as the experienced states and failed trials suggested by the reviewer. However, as this requires substantial extra engineering effort and ideation, we think it’d be more suitable to study these memory variants in another standalone work.\n\n**Generalization across tasks**\n> We provide one case for this cross-task generalization behavior in Figure 6, where the agent learns to build new workflows on top of previously learned, easier ones. Nonetheless, our agent can choose *not to* refer to existing workflows and come up with new ones solely based on the action trajectory when solving new examples (as illustrated in Figure 2 pipeline), and therefore to some extent can avoid bias from irrelevant existing workflows.\n\n**Reliance on memory**\n> Our AWM pipeline induces workflows *only from trajectories that are predicted as correct*, and therefore is not expected to generate erroneous workflows which cannot solve tasks correctly. We also manually evaluated all the induced workflows (7.4 per website, so it is feasible to carefully check all of them; more detailed in Appendix A.3) and found them correct and useful. We cannot provide example workflows on each website in this response due to length limits, please find more examples of induced workflows in Appendix A.2. It would be an interesting follow-up study to examine and improve the quality of workflows and make AWM more effective, if possible.\n\n```\nSearch for a Product\n# To search for a product, I need to enter the search term in the search bar and press Enter.\nfill('search_bar_id', '{search_term}')\npress('search_bar_id', 'Enter')\n```\n\n```\nFind Directions Between Two Points\n# To find directions between two points, I will use the directions feature on OpenStreetMap. I will start by clicking on the \"Find directions between two points\" link.\nclick('149')\n# Next, I will fill in the \"From\" and \"To\" fields with the respective locations and select the mode of transportation.\nfill('158', 'FROM_LOCATION')\nfill('163', 'TO_LOCATION')\nselect_option('166', 'MODE_OF_TRANSPORTATION')\nclick('171')\n# I will then retrieve and send the calculated travel time and distance to the user.\nsend_msg_to_user('The estimated travel time from FROM_LOCATION to TO_LOCATION is ESTIMATED_TIME.')\n```", "claims_and_evidence": "From a performance perspective, the argument is well-supported, as the proposed search methods enhance agent performance. A clear upward trend is observed with increasing number of workflows in the memory. However, certain weaknesses in the experimental design remain, which will be discussed in the following sections.", "methods_and_evaluation_criteria": "The proposed methods and evaluation benchmarks are well-designed and reasonable.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "While the research direction is highly promising with significant potential impact, there are several weaknesses in the method and experimental design that should be addressed:  \n\n**Robustness of agent performance**: The memory construction appears to be heavily dependent on the selected tasks and workflows in the offline setting. While the paper provides a temporal analysis for the online setting which is valuable, it remains unclear how the method performs when the selected offline tasks vary significantly. Will the agent's performance remain stable across diverse task selections? This is an important question that should be addressed in the rebuttal.  \n\n**Memory formulation**: The current memory framework seems limited to subroutine workflows, but memory can extend beyond that. Experienced environment states themselves are crucial, as they enable agents to anticipate action effects and make more informed predictions. Also, failed trials can serve as valuable memory, helping the model recognize and avoid repeating mistakes. Expanding the scope of memory formulation beyond its current constraints could significantly enhance the approach.  \n\n**Generalization across task templates**: It is somewhat surprising that memory from one task type benefits another, given the potentially large differences in trajectories and elements involved. A more detailed analysis is needed to justify why this generalization works as demonstrated in Table 2. For instance, if no relevant workflows exist in memory, will the agent still retrieve one, leading to erroneous predictions? Further insights into this phenomenon would strengthen the paper.  \n\n**Reliance on memory**: As mentioned earlier, memory may not always be beneficial. Is there a mechanism for the agent to disregard unhelpful memory and rely on its own knowledge for planning? More broadly, the analysis could be greatly improved by detailing how the current pipeline operates in practice and how error cases are handled, rather than focusing solely on the textual representation of workflows.", "relation_to_broader_scientific_literature": "Agent workflow memory has the potential to become a popular paradigm for enabling agents to adapt to new environments, an area that has been under-explored in prior work. The paper provides a strong formulation of this paradigm.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "a9Xe7FMvZD", "reviewer": "Reviewer_Uevr", "summary": "Motivated by the ability of humans to flexibly solve complex tasks by learning reusable workflows from past experiences, this paper introduces Agent Workflow Memory (AWM), a training-free paradigm designed to enhance the performance of language model-based agents in solving long-horizon, complex tasks. The core of AWM is an LM-based module that extracts reusable workflows from an agent's experience trajectories. These workflows serve as structured memory, allowing the agent to leverage past knowledge to guide future decision-making. The proposed method is evaluated on two web-based benchmarks, demonstrating strong cross-template and cross-task generalization. Additionally, the paper explores the impact of online/offline settings and different workflow representations on overall performance.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for recognizing the soundness of our method, the general applicability in online and offline settings, and the comprehensiveness of our experimental analysis! \n\n**W1. Experiments tailored to web agents**\n> We fully agree that AWM can be similarly applied to other agentic tasks such as robotics or tool use. We conducted very comprehensive experiments with two major web agent benchmarks emphasizing varied correctness and generalization aspects, which we believe are sufficient contributions to this work and prove AWM’s effectiveness; as many influential works on agents focus on one domain as well [1,2]. Nonetheless, applying AWM to other tasks such as robotics would require substantial engineering effort (setup hardware and software), which we think would be more suitable to explore in standalone future works.\n\n**W2. LM introduces bias into workflow construction and judgment processes**\n> For workflow construction, we have compared LM-based induction and rule-based induction (no LM involved) in Section 4.1 & Table 5. The rule-based method directly adopts content from the example without introducing any bias from the model, but their comparable scores show that LM does not introduce any “negative bias” into the workflows.\nFor the judgment process, we have compared using LM-based evaluation and ground-truth evaluation on WebArena in preliminary studies, and found that LM-based judgment leads to a higher average success rate. We analyzed results and found ground-truth evaluations to be overly strict and disagree more with human correctness judgments; whereas LM is more flexible and makes more accurate judgments. For example, ground-truth evaluation checks if the final state url exactly matches the annotated one (“xxx.map.com/cafe-near-cmu”), while the agent may land on “xxx.map.com/cafes-near-cmu” that searches the plural “cafes” instead of “cafe” but still get the correct results; ground-truth evaluators will inaccurately judge the result to be wrong, but LM-based evaluator and humans often still mark it as correct.\n\n**W3 & Q4. Compare to using raw experiences**\n> We *have* compared AWM to using raw experiences on both datasets.\nFor Mind2Web results in Table3, the Synapse method is conditioning on retrieved raw examples for task-solving, and AWM outperforms it by 13.1% in success rate.\nFor WebArena results in Table4, the rule-based induction is a stronger version of using raw examples, in that we further remove the invalid steps in raw examples to improve their quality. As AWM achieves comparable results to this stronger raw-example-using baseline, we expect AWM to perform better than using the original raw examples.\n\n**Q1. Online evaluation granularity and quality**\n> The LM-based evaluation is applied to the overall outcome, i.e., on the full trajectory. We induce “correct” workflows from trajectories that are predicted as correct, preventing incorrect trajectories from being converted into workflows. We made this design choice to ensure the quality of workflows and the effectiveness of the entire pipeline. It would be an interesting extension of our work to further consider adding incorrect trajectories for workflow construction, but it will necessitate stronger trajectory evaluators and design changes in the pipeline to make it more effective in improving end success rate.\n\n**Q2. Can AWM generalize to ALFWorld and SciWorld**\n> Yes, our AWM method generally applies to agent frameworks as everything is built on top of general agent components. While it depends on the task and domain overlap between our experimented benchmarks (WebArena, Mind2Web) and other datasets (ALFWorld, SciWorld) to tell if workflows produced in our experiments can be directly used to solve other datasets, it would be interesting to explore AWM performance on ALFWorld and SciWorld in future work.\n\n**Q3. AWM Requirement on Underlying Model Capabilities**\n> AWM does require the LM to have certain capabilities in task-solving, trajectory evaluation, and workflow induction. We have experimented with LLaMa 70B and Deepseek-R1 32B and find they benefit less from AWM — achieving 7.9→6.2 and 13.5→11.6 success rate before→after applying AWM – we will reference these results in the revised paper. Nonetheless, our main goal is to showcase this promising self-adaptive agent framework as agents get increasingly stronger.\n\n[1] Liu, Evan Zheran, et al. \"Reinforcement learning on web interfaces using workflow-guided exploration.\" \n\n[2] Wang, Guanzhi, et al. \"Voyager: An open-ended embodied agent with large language models.\"", "claims_and_evidence": "The paper claims that learning reusable workflows from past experiences can guide agents in solving complex, long-horizon tasks, thereby improving their performance and that AWM operates effectively in both online and offline settings. The supporting evidence for these claims is thoroughly reported in the experimental results, demonstrating significant improvements on WebArena and Mind2Web benchmarks. \\\nHowever, the claim that AWM can quickly learn and adapt across all tasks is only validated on web-based tasks, lacking evaluation on broader agent tasks, which may limit the assessment of its generalization capability.", "methods_and_evaluation_criteria": "The proposed method and evaluation criteria are well-aligned with the problem of improving LLM-based agents' performance on long-horizon, complex tasks.", "theoretical_claims": "The paper primarily focuses on an empirical evaluation of the proposed method AWM rather than presenting formal theoretical claims. As there are no mathematical theorems or formal derivations included, there are no proofs to verify.", "experimental_designs_or_analyses": "I checked the soundness and validity of the experimental designs and analyses presented in the paper, particularly focusing on the evaluation of Agent Workflow Memory (AWM) on the WebArena and Mind2Web benchmarks. The use of these benchmarks provides a realistic and diverse testing ground, and the comparison with SOTA baselines is a strong validation of the method's improvements. The analysis is relatively detailed, covering AWM's generalization, the differences between online and offline settings, and the impact of workflow representations.", "relation_to_broader_scientific_literature": "The paper's key contributions are closely related to several areas in the broader scientific literature, particularly in LLM-based agents and memory-augmented learning. The study builds on prior work in web-based agents (WebArena and Mind2Web [Deng et al., 2023]) that evaluate LLMs on digital task execution. The idea of integrating structured memory for long-term adaptation is related to the prior work on LLM memory mechanisms.\nUnlike methods that use retrieval, AWM autonomously induces structured workflow memories, allowing for more flexible and scalable adaptation.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "Strengths:\n\n1. The proposed method is training-free, minimizing computational overhead and making it more efficient compared to methods requiring extensive fine-tuning.\n\n2. AWM is applicable in both offline and online settings, allowing it to function effectively with or without high-quality annotated data, thereby reducing annotation costs.\n\n3. The experimental analysis is comprehensive, covering AWM's generalization capabilities, the differences between online and offline settings, and the impact of workflow representations on performance.\n\nWeaknesses:\n\n1. The experiments are limited to web-based agent tasks, lacking evaluations on other types of agent tasks (e.g., robotics, API-based interactions), which may restrict the assessment of AWM's broader applicability.\n\n2. The LM-based workflow construction and LM-based judgement of experiences in the online setting may introduce inherent biases from the language model, yet the paper does not analyze their potential impact.\n\n3. The approach shares similarities with few-shot learning, limiting its novelty. Additionally, the paper does not explore the direct impact of using raw experience versus structured workflows on agent performance, which could provide further insights into the benefits of workflow induction.", "other_comments_or_suggestions": "See Question", "questions_for_authors": "1. In the online setting, is the LM-based evaluation of experience applied to the overall outcome or each step of the process? Additionally, I would like to understand the impact of incorrect experiences being converted into workflows on the final test results. Could you provide experimental results where different proportions of erroneous workflows (e.g., 20% error, 50% error) are introduced?\n\n2. The experiments in the paper focus solely on web-based agent tasks. Can AWM be effectively applied to other agent environments, such as ALFWorld or SciWorld? Additionally, I am curious whether workflows learned from web-based tasks can generalize to these environments.\n\n3. I noticed that the experiments are conducted using high-quality OpenAI models. To what extent does the capability of the underlying LLM affect the effectiveness of AWM? I would like to see results using more open-source models (e.g., LLaMA, Qwen).\n\n4. How does directly using raw experiences as few-shot demonstrations compare to using AWM in terms of test performance? A comparison between these two settings would help clarify the advantage of workflow-based memory.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ib6bdSvsS0", "reviewer": "Reviewer_mwTP", "summary": "The paper introduces Agent Workflow Memory (AWM), a method designed to improve the performance of language model-based agents in long-horizon, complex tasks such as web navigation. AWM enables agents to learn reusable workflows from past task experiences and integrate them into memory for guiding future actions. Unlike conventional methods that rely on static training examples or in-context learning, AWM dynamically induces workflows that capture frequent sub-routines across tasks, allowing agents to generalize better and solve tasks more efficiently.\n\n### Update After Rebuttal\nThe paper presents unclear and problematic methodlogy for workflow induction and **missing key benchmarks** in the evaluation.. Thus, I keep my score.", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Claims and Evidence 1: AWM Improvement May Come from Structured Nature of Actions**\n> First, we argue that WebArena and Mind2Web queries are realistic and human-annotated without intentionally injecting superficial structures, so we believe that experimenting on them is well-motivated. Our method depends on structures that are shared **across trajectories** (e.g., adding an item to a cart, searching for a term), so individual trajectories need not be repetitive.\nMeanwhile, datasets for other tasks, e.g., embodied/robotics, actually often have more superficially structured queries (e.g., “harvest 1 dirt” and “harvest 8 dirt” on Minedojo), so we may not fully agree that switching to other tasks would be a perfect solution for alleviating learning superficial task structures. \n\n**Claim and Evidence 2 & Experimental Design 1: Dependency on Prompting Strong LLMs**\n> Although we experiment with strong GPT models, we do achieve very substantial improvements (over 50%) compared to using the vanilla GPT-4o model. For this reason,we would like to argue that AWM is effective, instead of having “limited” “methodological advancement”.\nOne prerequisite for AWM (and many other autonomous agent frameworks) is that the LM backbone needs to have reasonable abilities in task-solving. Our main focus in this work is to establish this promising self-adaptive agent framework. It would be interesting future work to investigate the prerequisites for LM capabilities or other aspects for AWM  to show effect.\n\n**Ambiguity in the Definition of Workflow Induction**\n> As we described in lines 141-161, we input several examples into GPT model and ask it to generate workflows, which are different from input examples in two aspects: (i) they have sub-trajectory level granularity and thus are more reusable than full-trajectory examples, and (ii) they abstract away example-specific contexts and replace them with descriptive variables (e.g., “New York” to “{destination city}”) to facilitate generalization across tasks.\n\n**Unclear Control over Workflow Granularity**\n> We agree that since we mostly rely on LMs to come up with workflows, we have limited control over the granularity generated by the LM. The alternative approach is rule-based induction, which hardly produces any workflows at all given the dynamic environment nature of web navigation tasks.\nEmpirically, we have carefully analyzed the quality of all induced workflows in Appendix A.3, about the number of workflows induced, their coverage on test queries, their functional overlap, etc. Please find more analysis in Appendix A.3 and example workflows in Appendix A.2. \nThroughout the analysis, we find the workflows have reasonable granularity, as we prompted the LLM to generate sub-trajectories and abstract away example-specific contexts. \n\n**Benchmark missing: WebVoyager, MiniWob++, WorkArena**\n> WebArena and Mind2Web are more widely used. Webvoyager may raise safety concerns as it operates on real-world websites; MiniWob++ is simplified and agent performance is not saturated (over 95% success); WorkArena is application-specific and largely follows the WebArena setup. We are interested in seeing future works experimenting with AWM with these more specialized benchmarks.\n\n**O1. Definition of Memory Size**\n> In the offline setting, we learn workflows from training examples and keep the memory size fixed in the later inference process. In the online setting, the memory size is continuously growing as the agent solves more tasks and induces more workflows.\n\n**O2. Unclear observation and action space**\n> As mentioned in lines 189-195, we adopt the browsergym framework and use the webpage accessibility tree as the agent observation. We adopt the WebArena action space supported by BrowserGym. We will add the list of actions to the revised paper version.\n\n**O3. What is the “base memory”**\n> Base memory refers to the original agent memory before adding any induced workflows. Implementation-wise, this contains instructions for the model to follow the user query and solve the tasks. We provide the exact prompt, a.k.a., the base memory, in Appendix A.1.", "claims_and_evidence": "1. The claim that “AWM enables agents to learn and reuse workflows, improving task success rates” lacks sufficient justification. The reported improvements could be task-specific rather than a generalizable advantage of AWM. Given that web browsing tasks inherently contain repetitive action sequences, AWM’s success may stem more from the structured nature of these tasks rather than from a fundamental ability to generalize workflow induction across diverse domains. A more rigorous evaluation on non-web tasks or across varied domains would strengthen this claim.\n2. AWM is fundamentally a prompt-based approach that depends heavily on the underlying base LLM. This reliance raises concerns about its adaptability to different LLM architectures. If AWM’s effectiveness is largely dictated by the capabilities of the LLM it prompts (e.g., GPT-4 vs. a weaker model), then its contribution as an independent methodological advancement is limited. Without demonstrating robustness across multiple LLMs, the method’s general applicability remains uncertain.", "methods_and_evaluation_criteria": "Issues in Section 2.3: \"Inducing and Using Workflows\"\n\nSection 2.3 presents an unclear and problematic methodology for workflow induction.\n\n1. **Ambiguity in the Definition of Workflow Induction**\n- Lines 138–140 state: *“The workflow induction module operates by taking in E and producing a set of workflows, as …”* but do not clearly define how this transformation occurs.  \n- The notation suggests that \\( I \\) (the induction module) takes a **set of experiences** \\( E \\) as input and produces a **set of workflows** \\( W \\). However, the output structure (as described in Line 140) appears **identical** to the input experiences, maintaining the same elements (\\( d \\) and \\( P \\)).  \n- This lack of transformation raises concerns: Is the module simply **repackaging** past trajectories instead of **deriving** reusable workflows? If not, how does it extract structured workflows from raw experiences?\n\n2. **Unclear Control Over Workflow Granularity**\n- The method does not specify **how it determines the appropriate granularity** of induced workflows.  \n- A workflow could capture a **low-level skill** (e.g., searching for an item in a menu) that is **widely reusable** across tasks, or it could represent a **high-level task** (e.g., booking a hotel), which is **more specific** and less transferable.  \n- If the induction process **fails to regulate granularity**, it risks either:  \n  - Extracting **too generic** workflows that do not meaningfully aid decision-making.  \n  - Inducing **overly specific** workflows that are not broadly reusable, even if stored in memory.  \n- The paper should clarify **how the induction module balances this trade-off**, potentially by leveraging hierarchical workflow structures or controlling abstraction levels during induction.\n\nWithout addressing these issues, the workflow induction process remains **ill-defined and difficult to evaluate**, limiting the interpretability and generalizability of the proposed method.", "theoretical_claims": "There're no theoretical claims in this paper.", "experimental_designs_or_analyses": "The overall results appear promising, but there are significant concerns regarding the **generalization of AWM beyond the specific LLM used** and **missing key benchmarks** in the evaluation.\n\n1. **Dependence on Heavy Prompting and LLM Generalization**\n- The entire method is **heavily reliant on prompting**, making its **generalization to other LLMs uncertain**.  \n- The paper does not explore **how AWM performs with weaker LLM architectures** (e.g., LLaMA or other open-sourced models).  \n- A broader evaluation across different LLMs would provide **stronger evidence of its robustness**.\n\n2. **Missing WebVoyager Benchmark**\n- The results focus on **two web navigation tasks**: WebArena and Mind2Web. However, **WebVoyager**, another widely used web-based agent benchmark, is **not discussed** or evaluated.  \n\n3. **Incomplete Benchmark Comparisons in Table 1**\n- **Table 1** presents **results primarily compared to BrowserGym**, but **misses two important web agent benchmarks**, which are WorkArena and MiniWob.\n- Both **WorkArena and MiniWob** are **tasks where BrowserGym excels**, making it unclear how AWM would compare in **scenarios beyond the two selected benchmarks**.\n- **A more comprehensive benchmark comparison** is needed to properly situate AWM’s performance **relative to existing methods**.", "relation_to_broader_scientific_literature": "This paper relates to the LLM agent, especially browser-based agent applications.", "essential_references_not_discussed": "This paper misses the following benchmarks:\n1. WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models, arXiv:2401.13919\n2. MiniWob - Humphreys, Peter C., et al. \"A data-driven approach for learning to control computers.\" International Conference on Machine Learning. PMLR, 2022.\n3. WorkArena - Drouin, Alexandre, et al. \"Workarena: How capable are web agents at solving common knowledge work tasks?.\" arXiv preprint arXiv:2403.07718 (2024).", "other_strengths_and_weaknesses": "A major concern with the paper is lack of clarity, as it introduces several key concepts without proper definitions or explanations. This makes it difficult to fully understand the method and its implications.\n1. One unclear aspect is the definition of agent memory size. It is unclear whether the memory scales dynamically or remains fixed. If it is fixed, the paper does not explain how its size was determined. More details on memory allocation and constraints would help clarify its practical feasibility.\n2. Another issue is the lack of clear definitions for states, observations, and actions in the agent's design. The paper does not specify whether states are represented as webpage snapshots, structured representations, or tokenized text inputs. It is also unclear how observations differ from states and how they integrate with the memory system. Furthermore, the paper does not clarify what action space the agent operates in. For example, it is unknown whether the agent uses predefined UI actions like \"CLICK\" and \"SCROLL\" or if it generates free-text commands.\n3. The term \"base memory,\" mentioned in line 59 on the right, is also ambiguous. The paper does not define whether \"base memory\" refers to the default knowledge the agent starts with, the workflow memory that stores reusable subroutines, or a temporary buffer for task execution. It is also unclear whether there are other types of memory besides base memory and, if so, how they differ.", "other_comments_or_suggestions": "Typos in L115 Right, should be $\\mathcal{W}$ not W.", "questions_for_authors": "Please check comments above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jI0irTPcvn", "reviewer": "Reviewer_azRW", "summary": "This paper proposes the Agent Workflow Memory (AWM) that can summarize and abstract common workflow experiences from previous trajectories. However, it seems too similar to the series of Reflexion works. The author should discuss the significant contributions comparing with these works.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the feedback and for pointing out these relevant works. We list the core differences between AWM and these works below, and will add these more detailed discussions in our revised paper version!\n\n**Difference from Reflexion**\n> 1. Content: AWM learned procedural knowledge on how to solve specialized, reusable tasks, i.e., only correct trajectories; Reflexion receives errors and comes up with plausible (erroneous, correct) solution pairs.\n2. Context window: AWM carries reusable procedural knowledge across multiple tasks experienced over time, therefore could leverage and benefit from the procedural similarity across tasks. Reflexion only maintains learned reflections within the current single-task solving session.\n\n**Difference from Expel**\n> 1. Requirement on training data: Expel requires learning from training examples; AWM not only supports learning from training/offline tasks, but can also be directly applied online/testing *without the need for any training data* beforehand.\n2. Expel stores raw trajectories in memory (similar to Synapse below); where our AWM stores sub-trajectory level, context-abstracted procedures, which have shown to be more reusable and beneficial in our experiments (Table 3 and Table 4).\n\n**Difference from Synapse**\n> 1. Full-trajectory vs. Sub-trajectory: AWM proposes to learn reusable procedures that are often part of the trajectory for a given task; in other words, a complex, long-horizon task usually needs to concatenate multiple workflows to solve it correctly. Synapse only remembers the full trajectory, but is often less reusable for other tasks (as the example in lines 147-161).\n2. Dependent vs. Independent on example-specific context: Synapse records specific contexts to individual examples, e.g., “Flight from New York to Pittsburgh”; while AWM abstracts away those variable contexts to, e.g., “Flight from {departure} to {destination}”, so that it could be more easily reused to apply to other examples such as booking a flight from Chicago to Toronto (also refer to the example in lines 147-161).", "claims_and_evidence": "It claims the proposed method can abstract common workflow from past experiences and apply it to guide future interactions. The experiments are related to this claim.", "methods_and_evaluation_criteria": "I think the proposed methods are too similar to the series of Reflexion works, such as [1-3]. The author should discuss the significant contributions compared with these works. \n\n[1] Shinn, Noah, et al. \"Reflexion: Language agents with verbal reinforcement learning.\" Advances in Neural Information Processing Systems 36 (2023): 8634-8652.\n\n[2] Zhao, Andrew, et al. \"Expel: Llm agents are experiential learners.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 38. No. 17. 2024.\n\n[3] Zheng, Longtao, et al. \"Synapse: Trajectory-as-exemplar prompting with memory for computer control.\" arXiv preprint arXiv:2306.07863 (2023).", "theoretical_claims": "No theoretical claims in this paper.", "experimental_designs_or_analyses": "The author conducts experiments on WebArena and Mind2Web. The domains are limited.", "relation_to_broader_scientific_literature": "Yes, very relevant.", "essential_references_not_discussed": "[1] Shinn, Noah, et al. \"Reflexion: Language agents with verbal reinforcement learning.\" Advances in Neural Information Processing Systems 36 (2023): 8634-8652.\n\n[2] Zhao, Andrew, et al. \"Expel: Llm agents are experiential learners.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 38. No. 17. 2024.", "other_strengths_and_weaknesses": "See above.", "other_comments_or_suggestions": "See above.", "questions_for_authors": "Could you discuss the significant contributions compared with these works?\n\n[1] Shinn, Noah, et al. \"Reflexion: Language agents with verbal reinforcement learning.\" Advances in Neural Information Processing Systems 36 (2023): 8634-8652.\n\n[2] Zhao, Andrew, et al. \"Expel: Llm agents are experiential learners.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 38. No. 17. 2024.\n\n[3] Zheng, Longtao, et al. \"Synapse: Trajectory-as-exemplar prompting with memory for computer control.\" arXiv preprint arXiv:2306.07863 (2023).", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces AWM, a framework that enables agents to extract reusable workflows from past experiences and use them to guide future actions in long-horizon tasks like web navigation. The motivation is intuitive — procedural memory helps humans generalize. The empirical results on WebArena and Mind2Web are solid, showing significant gains over baseline agents.

The empirical results on WebArena and Mind2Web are solid, showing significant gains over baseline agents. However, several concerns remain unresolved. First, the workflow induction process is underspecified — it’s not clear how generalizable or robust the abstraction mechanism is, especially in terms of granularity control and reproducibility. There’s a lack of clarity on whether the method truly generates reusable workflows or just repackages prior trajectories. Additionally, the memory management (size, updating policy, etc.) and the agent’s action/observation definitions are not well-articulated. Lastly, the method feels adjacent to prior work in Reflexion, Expel, and Synapse. The authors address these differences, emphasizing AWM’s focus on procedural subroutines over error-based memory or raw trajectory replay. But still feels like an incremental advance in a crowded space (although I know this work was released a while back on Arxiv already, if we take that into account).
Decision: Accept (poster)