Hierarchical Prompting Assists Large Language Model on Web Navigation
Authors: Abishek Sridhar
Robert Lo
Frank F. Xu
Hao Zhu
Shuyan Zhou
Venue: EMNLP 2023 Findings (2023)
Keywords: LLM Prompting
Web Navigation
TL;DR: We proposed a simple hierarchical prompting approach that improve Large Language Models (LLMs) performance on web-navigation task.

Abstract: Large language models (LLMs) struggle on processing complicated observations in interactive decision making. To alleviate this issue, we propose a simple hierarchical prompting approach. Diverging from previous prompting approaches that always put the full observation (a web page) to the prompt, we propose to first construct an action-aware observation which is more condensed and relevant with a dedicated Summarizer prompt. The Actor prompt then predicts the next action based on the summarized history. While our method has broad applicability, we particularly demonstrate its efficacy in the complex domain of web navigation where a full observation often contains redundant and irrelevant information. Our approach outperforms the previous state-of-the-art prompting mechanism with the same LLM by 6.2% on task success rate, demonstrating its potential on interactive decision making tasks with long observation traces.

Hierarchical Prompting Assists Large Language Model on Web Navigation
Abishek Sridhar"
Chi-Fan Lo" Frank F. Xu Hao Zhu
Shuyan Zhou
School of Computer Science, Carnegie Mellon University
{abisheks, chi mail, fhifanl, zhuhao, shuyanzh)@cs cmu. edu
Abstract
Product description
Raw Observation
Aluminum Free: Natural Deodorunts are notocious for not working.
Large language models (LLMs)) struggle pro- cessing complicated observations in interactive decision making tasks. To alleviate this issue, we propose simple hierarchical prompting ap- proach. Diverging from previous prompting approaches that always put the fill observa- tion (e.g., web page) to the prompt, we pro- pose to first construct an action-aware obser- vation which is more condensed and relevant with a dedicated SUMMARIZER prompt. The ACTOR prompt then predicts the next action based on the summarized observation. While our method has broad applicability, we partic- ularly demonstrate efficacy in the complex domain of web navigation where full obser- vation often contains redundant and irrelevant information. Our approach outperforms the previous state-of-thes the--rt prompting mechanics by 6.2% on task success rate, demonstrating its potential interactive decision making tasks with long observation traces.
[button] assocted wood secnts [button]
Summarizer
[button] citnus [button]
[button] [button]
[button] travel set (4-pack) [button]]
[buttos] 3 ounce (pack of [button]
Summmaried
[botton] 5-ounce (2-pack) [button) [botroe] family [button]
Observation
[buston] cicnes [busion]
[button] mango [button]
Size
[button] 3 curce (pack [button]
[button] 3-ounce (2-pack) [button]
Figure I: Asi Prompting. In each round, the SUM- MARIZER will produce a condensed observation given observation and the ACTOR'S previous action. The ACTOR will produce the next action based on the more concise summarized observation and the interaction his- tory. The right side shows the raw observation and the summarized observation for the instruction "Find me a small bottle fruit deodorant".
through few-shot in-context examples (Ahn et al., 2022). Previous rescarch has focused on various techniques for shaping the action space, such as structured representations of actions through pro- grams (Zhou et al., 2021; Liang et al., 2022; Singh et al., 2022), and promoting synergy between rea- soning and acting (Yao et al., 2022b). However, little attention has been paid to effectively encod- ing the states, which is particularly crucial when the interacting environment is diverse and lengthy.
1 Introduction
our daily lives, we often encounter tasks such household duties (Shridhar et al., 2020) and web navigation (Shi et al., 2017; Zhou et al., 2023) that necessitate interactive and sequential decision- making. These tasks require us take actions (e.g.,
entering search query) based on both the state of the environment and the specific objectives (e.g.,
buy a shirt). There has been growing interest au- tomating these decision makinn tasks using natural language commands (Yang et al., 2023).
This work aims at optimizing the state obser- vation with prompting method. We propose ASH Aatoo=Suummarzee--lirrrrciicii prompting as in Figure 1. Our approach has two key components: (1) A SUMMARIZER that takes a raw observation from the environment and produce sentation that is more meaningful to the goal. It achieves this by learning from dedicated prompt that demonstrates the expected summarization different scenarios. Figure 1, the instruction requests a small bottle of deodorant with a fruit
Large language models demonstrate their po- tential performing interactive decision mak- ing tasks, owing to their capacity for encoding vast amounts of real-world knowledge. One com- mon approach for utilizing LLMs in these tasks is
repre-
Eqqual contribution Corresponniig author
'Code is availahle at https://github.com comm robert1000 1at11000//ashaaa paatttttta/s----ptiinggggg

---

scent. The current product includes suitable con- figurations that meet this requirement. However, there are unnecessary details like product descrip- tions and distracting options such as ture-volumme bottles. The summarizer filters out these irrelevant contents, retaining only the relevant information for further processing. (2) An ACTOR that takes the trajectory history (i.e., observations and actions) and produce the next action. The hierarchical mod- ularized design of ASH effectively reduce the heavy reasoning burden by avoiding dumping all infor- mation to single prompt. The SUMMARIZER ASH could effectively remove irrelevant infor- mation, and provide more concise observation representation. This representation aligns with the demonstration in the context for the actor to make precise predictions.
Act ion: iik (BC78GARC1J)
rust ions Find me a ounce bottle of of bright citrus deodcrant
Observation:
scent
[button] assorted scents [button]
[button] bright citrus [butson]
size
[button] travel set (4-pack) [button]
[button] 3 ounce (pack of 1) [button] [button] 3-ounce (2 (2-pack) [batton]
Information:
Let us thick step-by-step. First, I need to let the aster know if this is the correct product. Then, I need to filter out options that does not match to make it easier for the actor to choose from
We apply ASH on web navigation with spe- cial focus on purchasing products on a real-world scale E-commerce simulator Webshop (Yao et al., 2022a), since it is the only relevant existing dataset with a complex observation space to the best of our knowledge. Our ASH significantly outperforms the previous state-of-the-arr approach (Yao al., 2022h) by 6.8% task success rate (29% relative gain). These results show the promise of ASH in addressing the challenges of comprehensive history and complicated states in real-world interactive de- cision making tasks.
scent [button] bright citrus [button] size
[button] bright citrus [butson] [button] ounce (pack of 1) [button] [button] 3-ounce (2-pack) [batton]
Figure 2: An example in the SUMMARIZER prompt. The history information in blue (*..." indicates the omission of the full content); CoT instruction for the SUMMARIZER is in green. The SUMMARIZER'S output is black.
red; the raw observation is
and predicts a, Formally, ASH decomposes the probability of generating at to two components:
2 The Web Navigation Task
P 2, 2 a,0
this section, we formally define the task of web navigation. Given web environment E, initial state So with observation (typically a web page). the goal of an agent is to reach a specific environ- ment state s* defined by natural language u by per- forming a sequence of actions. At time I, the agent needs to decide the action a based on interaction history up to (HI, = {oo, a1, O1..... at 1, ot }). Once a stop action is predicted, the score of the task is measured by comparing the final state & with s* with similarity function f(8, s*) which ranges from zero to one.
P (0((10-1,,0) P (aaa|Hi-22
(stateless) SUMMARIZER
(stateful) ACTOR
3.1 SUMMARIZER
The goal of the SUMMARIZER is to generate a more concise observation that only encodes the relevant information. It identifies and removes irrelevant information guided by the provided in-context ex- amples. An example of the SUMMARIZER prompt is in Figure 2 and the full prompt is listed in Ap- pendix C.I and C.2.
There are several key information provided the prompt for SUMMARIZER: (1) the previous action and the instruction (in red) that gives SUM- MARIZER hints how and why does the agent navigate to the current state. (2) raw environment observation (in blue) at current step that needed be summarized. In web navigation task, this cor- responds to the contents of the current web page. (3) a customized step-by-sted instruction (Wei et al.,
3 ASH Prompting
As summarized in Figure 1, ASH comprises of stateless SUMMARIZER that takes action 0:-1 and observation o, and then outputs condensed observation of that semantically comprehensible by the ACTOR Then, the stateful ACTOR takes up-
dated history Hi = {o), 01, oz a2,-... a4-1.11 of
2

---

Instruction: Find me a ounce bottle of of bright citrus deoderant
some NL descriptions (e.g, "Find me black e- reader with price lower than 270 dollars"). In this benchmark, the agent can perform three ac- tions: (1) search with text-based query (2) click[...] any interactable web element (3) think[...] the current situation and make further movements. The think action, which does not change the state, is adopted from REACT (Yao et al., 2022b). Webshop is challenging for intelli- gent agents due its large-scale and diverse prod- uet catalog, consisting of 1, 181, 436 unique prod- ucts, each with its own distinct features, resulting in a highly complex and varied state space."
History:
Actlon 1:
search/3 ounce bright citrus
deodorant sensitive skin]
Observation 1:
Search results for keyword "3 ounce..." [B078CKRC1J]
Bright citrus Deodorant...
$10.99
Action 4:
click [3 ource (pack of 1)]
Observation 4: You have clicked 3 ounce (pack of 1).
Task Score Calculation We follow (Yao et al., 2022a) to calculate the task scores. If trajectory ends with buying a product, will be assigned a score between zero and one, denoting the percent- age of required attributes satisfied by the product. A score of 1.0 indicates successful trajectory, while 0.0 marks failure. However, as special case, if trajectory either produces & consecutive invalid actions exceeds the max length, it will be terminated and assigned a score of 0.0.
Action 5: click [Buy Now]
Figure 3: An example in the ACTOR prompt. The NL instruction is in red; the history is in blue. The ACTor's output black.
2022) on how to summarize the current state (in green). Importantly, we design specialized exam- ples tailored to different scenarios to encourage satile reasoning. For example, as in Figure 2, the instruction outlines how to process product page. Specifically, the SUMMARIZER is tasked to verify the desirability of the product, followed by filter- ing out any irrelevant options. More instructions on improving search query, compressing search results and others can be found in Appendix C.I. In this way, the summarized states can be more customized towards the current scenario.
Model and Hyperparameters We experiment with CODE-DAVINCI-002" (Chen et al., 2021) and GPT-3.5-TURBO" (OpenAI, 2022). We use greedy decoding with max token of 500. We terminate a trajectory after 20 steps (the average number of steps Webshop is 6.5), or if the agent issues an invalid action (e.g.. click a button that does not exist) for five consecutive steps.
5 Results
The main result is given in Table 1. ASH con- sistently outperforms the previous state-of-the-art prompting mechanism REACT with different base models. More specifically, with CODE-DAVINCE 002, ASH brings 6.8% gain on success rate over REACT (29% relative gain) and improve the av- erage score by over 3.3%. ASH demonstrates a more significant improvement when used with GPT- 3.5-TURBO. A primary reason for this difference
3.2 ACTOR
We follow existing works to design the ACTOR prompt. Each example is consist of the NL instruc- tion, the interaction history up to the current time step f. The history is presented as an alternating sequence of actions and summarized observations. The goal of the ACTOR is produce the desired action for the current step. An example of ACTOR is listed in Figure 3 and the full prompt is listed in Appendix C.3 and C.4.
"We utilized the text-rich settings of the Wehshop the entire product space and sampled consistent 500 trajectories for experiments. Unlike (Yao al., 2022b), we kept the wehpage complexities and did not restrict search results to the top three. The trajectory generation code is in Appendix F.
4 Experiment
Dataset We test ASH on Webshop (Yao al.. 2022a), a real-world scale E-commerce bench- mark where the agent interacts with an Amazon- style website to purchase products described by
"W found that the free CODE-DAC OODE-DAVINCI-O0 yields stronger performance than the paid TEXT-DAVINCI-OO1 hence we used the former for our experiments. The comparison shown in Appendix
"We used GPT-3.5- N-TT3.5-TURBO-06133
3

---

is the context length limitations of 4k in GPT-3.5-
14
12.7
TURBO (CODE-DAVINCI-002 offers a longer con- text window of 8k). ASH summarizes observa- tions and effectively encodes history within this limited context budget. In contrast, REACT often surpasses the context length, leading to failures. This trend is unchanged when using GPT-3.5- TURBO actor with no in-context examples, proving the method' insensitivity to number of in-context examples. These results demonstrate the effective- ness of ASH on performing web navigation tasks. We illustrate REACT and ASH on sample trajec- tory in Appendix and analyize the behavior of SUMMARIZER qualitatively in Appendix E.
13
12
10.4
REACT
ASH
(a)
30
27.4
28
26
23.4
24
22.2
22
DIRECT
REACT
DIRECT+ASH
(b)
Figure 5: Top: percentage of failures due repeating invalid actions. Bottom: success rate (%) on different prompting mechanisms.
Method
Avg Score
Success (%)
53.4
23.4
ASHons-Downee--00
56.7
30.2
REACTorrs.-raae 27.3
3.0
generating more relevant and concise observations, which consequently enhances the ACTOR to make correct predictions in more complex tasks.
ASHumm3.5-ruu0
44.5
12.6
Table 1: Overall performance of REACT and our pro- posed method (AsI)) with CODE-DAVINCCI-002 and GPT-3.5-TURBO.
ASHI promotes valid action grounding LLMs suffer from hallucinations even when evidence is provided within the context (Liu et al., 2023). In interactive decision making tasks, these hallueina- tions manifest as the generation of invalid actions that cannot be executed in the current state, such as attempting to click a non-existent button. As shown in Figure 5a, 12.7% of failures in REACT occur due to the repetition of invalid actions, result- ing in termination because of the maximum step limit. In contrast, ASH exhibits relative reduction of 18% in this failure mode, effectively decreasing the failure rate 10.4%. Our hypothesis is that the presence of noisy and irrelevant context triggers hallucinations in LLMs, while providing a more concise and relevant context mitigates their impact.
Length Scare Didributtonn
259
200
190
259
Figure The average scores of traecctories grouped by the trajectory length, with the line plot representing score and the bar plot representing counts.
ASH mitigates the need of explicit reasoning Previous studies such as REACT and CoT sug- gests that explicitly performing reasoning would assist more accurate decision making. However. we argue that the requirement of such verbose rea- soning partially stems from LLM's inability to pro- cess complicated observations. To demonstrate this, we apply ASH to a simpler prompting mecha- nism that directly predicts the next action given the interaction history without the extra reasoning ac- tion (DIRECT). As in Figure 5b, we find that AsH would allow the naive prompting mechanism to out- perform the more sophisticated REACT approach by significant margin.
Performance vs. trajectory length Long- horizon tasks are challenging due to the difficulty keeping track of long histories. To analyze this problem, we plot the average score with respect to the length of the trajectories in Figure 4. We observe that ASH consistently yields higher aver- age score than REACT across different trajectory lengths. The performance gap is particularly large on examples with more than 11 steps. While RE- ACT only achieves an average score of 7.4, ASH increases the score to 38.2. This finding highlights the effectiveness of our SUMMARIZER module in
4

---

6 Related Work
tion thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or im- plied, of the Air Force Research Laboratory or the U.S. Government. This project was also partially supported by gift from AWS Al.
Prompting-based Decision Making Singh et al. (2022) and Liang et al. (2022) study the structured representation of actions. Instead of representing actions as linear sequence, they turn actions to components in programs encourage hierarchical structures (Zhou al., 2021). Yao al. (2022b) improve the action prediction by introducing an ad- ditional think step which verbally reasons about what needs to be done next. However, most works use the vanilla raw observations from the environ- ment (Ahn et al., 2022; Huang et al., 2022). RE- ACT Yao et al. (2022b) is the work that is closest to us. However, ASH differs from REACT mainly in two ways: (1) ASH breakdown the observation processing into two steps: first to condense observa- tions and then to act, while RFACT do the first step internally and only act. (2) ASH separate the de- velopment of ACTOR and SUMMARIZER prompts, allowing for greater enhancement, whereas REACT solely depends on the ACTOR prompt's efficacy.
Limitations
The primary aim of this paper is to introduce a new approach of prompting. We did not employ detailed manual or automatic prompt engineering for the ACTOR and SUMMARIZER prompts. An- other limitation is that we did not test our approach on more realistic benchmarks on web navigation, due to a lack of such benchmarks at the time we finished this work.
Ethics Statement
Our work is a novel prompting paradigm that could be applicable on various sequential planning tasks. To the best of our knowledge, this paradigm does not discriminate against or cause damage to any human. However, it is worth noting that au- tomating web navigation with models poses poten- tial risks such unintentionally access or scrape sensitive private information, leading to poten- tial breaches of privacy.
Multi-stage Prompting Multi-stage prompting is explored other tasks. (Liu et al., 2022) use multi-stage prompting model for knowledgeable di- alogue generation. They show prompting again on the previously generated knowledge can help the model to generalize to context outside of the knowl- edge base. Similarly, by applying different prompts during multiple stages, (Tan et al., 2021) show that this simple approach can effectively adapt pre- trained LMs to downstream tasks like translation.
References
Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, et al. 2022. Do as can, not as i say: Ground- ing language in robotic affordances. arXiv preprint urXiv:2204.0169.
7 Conclusion
We introduce ASH, a hierarchical method tailored for sequential decision-makirr makinn in web navigation using LLMs. Tested on the Webshop benchmark, ASH surpassed the strong prompting method Re- ACT in average score and success rate. Notably, ASH excels in complex tasks with lengthy trajecto- ries. It reduces LLMs' ballucinations and mitigates the need of explicit reasoning.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Ka- plan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao. Parker Barnes, Yi Tay. Noam Shazeer. Vin- odkumar Prabhakaran. Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengeheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia,
A Acknowledgement
We thank the anonymous reviewers for valuable feedback. This material is partly based on re- search sponsored in part by the Air Force Research Laboratory under agreement number FA8750-19. 2-0200. The U.S. Government is authorized to reproduce and distribute reprints for Governmen- tal purposes notwithstanding any copyright nota-
5

---

Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivami Agrawal, Mark Omernick, An- drew M. Dai, Thanumalayan Sankaranarayana Pil- lai, Marie Pellat, Aitor Lewkowyez, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang. Brennan Saeta, Mark Diaz. Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellsterr Douglas Eck, Jeff Dean, Slav Petrov, and Noah Fiedel. 2022. Palm: Scaling language mod- eling with pathways.
Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Ed Chi, Quoe Le, and Denny Zhou, 2022. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.1 19903
Sherry Yang. Ofir Nachum, Yilun Du, Jason Wei, Picter Abbeel, and Dale Schuurmans. 2023. Foundation models for decision making: Problems, methods, and opportunities. arXiv preprint arXiv:2303.04129.
Shunyu Yao, Howard Chen, John Yang. and Karthik Narasimhan. 2022a. Webshop: Towards scalable real-world web interaction with grounded language agents, arXiv preprint srXiv:220..02066
Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng. Jonathan Tomp- son, Igor Mordatch, Yevgen Chehotar, et al. 2022. Inner monologue: Embodied reasoning through planning with language models. arXiv preprint arXiv:2207.05008.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafram, Karthik Narasimhan, and Yuan Cao. 2022b. React: Synergizing reasoning and acting in language models. arXiv preprint Ttt1:22100030222
Shuyan Zhou, Frank Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Yonatan Bisk, Daniel Fried, Uri Alon, et al. 2023. Webarena: A realistic web environment for building autonomous agents. arXiv preprint arXiv:2307. 138544
Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. 2022. Code as policies: Language model programs for embodied control. arXiv preprint arXiv:220.00755..
Shuyan Zhou, Pengcheng Yin, and Graham Neubig. 2021. Hierarchical control of situated agents through natural language. arXiv preprint arXiv:2109
Nelson F Liu, Tianyi Zhang, and Percy Liang. 2023. Evaluating verifiability in generative search engines. arXiv preprint arXiv:2304.09%8 009448.
Zihan Liu, Mostofa Patwary, Ryan Prenger, Shrimai Prabhumoye. Wei Ping. Mohammad Shoeybi, and Bryan Catanzaro. 2022. Multi-stage prompting for knowledgeable dialogue generation. arXiv preprint arXiv:2203.08445.
OpenAI. 2022. Chatgpt: Optimizing language models for dialogue.
Tianlin Shi, Andrej Karpathy, Linxi Fan, Jonathan Her- nandez, and Percy Liang. 2017. World of bits: An open-domain platform for web-based agents. In In- lernational Conference on Machine Learning, pages 3135-3144. PMLR.
Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. 2020, Alfred: A bench- mark for interpreting grounded instructions for ev- eryday tasks. In Proceedings of the IEEE/CVE con- ference on computer vision and pattern recognition, pages 17740-10799
Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thomason, and Animesh Garg. 2022. Progprompt: Generating situated robot task plans using large language models. arXiv preprint 10X1:2209.11302.
Zhixing Tan, Xiangwen Zhang, Shuo Wang, and Yang Liu. 2021. Msp: Multi-stage prompting for making pre-trained language models better translators. arXiv preprint arXiv:2 100.660,
6

---

A Performance on other LLMS
We initially tested on smaller subset of 50 samples (out of 500) with contemporary models TEXT- DAVINCI-002 and CODE-DAVINCI-002. As evident table 2, we observed that in both methods (REACT and ASH) CODE-DAVINCI-O0) performs better than TEXT-DAVINCI-002, and a similar trend of ASH dominating REACT was evident in both models. Therefore, we proceeded to use CODE-DAVINCI-O0) in our results and analysis.
Due to the unavailability of PaLM model (Chowdhery et al., 2022) for public access and the mildly obscure implementation details of ReAct Webshop from the paper (Yao al., 2022b), re-implement our version ReAct on CODE-DAVINCI-ON and GPT-3.5-TURBO, and design prompts for ASH an analogous fashion to run our experiments.
Method
Avg Score
Success
REACT TEXT-DAVINCI-OOO
50.2
15.3
ASH + TEXT-DAVINCI-002
60.9
23.1
REACT CODE- AADE-DAVINCI-00
61.3
23.1
ASHI CODE-DAVINI--00
62.6
28.8
Table 2: Overall performance (%) of REACT and our proposed method (AsII) with TEXT-DAVINCI-002 on 50 random sampled trajectories.
B REACT vs ASH Prompting Trajectory
We an illustration of where REACT and ASH method differ in approaching instruction. While REACT hurriedly picks sub-optimal product without worrying about one desired property being off, ASH notices that the correct color is not in the product initially picked from the match information between instruction and description by the summarizer and proceeds to reformulate the query and buy the correct product. The actor also finds it easy to identify the optimal product based on the retention of fewer options by the summarizer, as we intuitively imagined. The example also demonstrates the effectiveness Ash in reducing the hesitation of the actor to take longer trajectories.
B.1 REACT
WebShop
Instruction:
Find me white item window coverings for living dining with color: color and size: 84"w 84"1, and price lower than 150.00 dollars
[earch]]
Action search white item window coverings for living room dining room] Observation
Instruction:
Find me white item window coverings living room, dining room with color: color and size: 84"w 84"1, and price lower than 150.00 dollars Back Search
Page (Total results: 50)
Next >]
[BOKLKNRMH]]
Gold Leaves Velvet Curtain Print Flannel Drape for Dining Living Room Rod Pocket Window Cover Light Filtering Home Decor Thermal Insulated Window Treatment Panel W39 x 1.88 Inch Emerald
$30.22
[B09L193GGF]
GY Blackout Window Curtain Red Stripe Wallpaper Living Room Bedroom Curtains 2 Panel Sets. Thermal Insulated Room Darkening Window Coverings Drapes 42W x 63L Inch
$30.87
[B09KV79XC9]
Blackout Window Curtain Panel White Dandelion 1 Pcs Curtain Coverings Rustic Wood
Barn Retro Farmhouse Grommet Top Window Drape for Bedroom Dining Living Room 52
x72in
7

---

$35.81
[B09KVTZTN4]
Blackout Window Curtain Panel Coastal Print Pes Curtain Coverings Conch Grommet Top Window Drape Bedroom Dining Living Room 52x84in
$40.5
[809KV8NNPP]
Blackout Window Curtain Panel Black and White Tulip Pes Curtain Coverings Flowers Blooming Grommet Top Window Drape for Bedroom Dining Living Room 52x90in
$41.82
[B0KKVBFSCK]
Blackout Window Curtain Panel Independence Day 4th of July Pcs Curtain Coverings American Flag Red White and Blue Grommet Top Window Drape for Bedroom Dining Living Room 52x52in
$31.17
[B09BFLLM28]
Marble Curtains Black and White Marhle Swirl Decor Living Room Bedroom Curtains 2 Panel Sets. Thermal Insulated Room Darkening Window Coverings Drapes 42x63inch (107x160cm)
$38 99
[B0OKV]HTT3)]
B Blackout Window Curtain Panel Valentine Day 1 Pes Curtain Coverings Black and White Stripe Love Grommet Top Window Drape for Bedroom Dining Living Room 52x63in
$32.07
[B07NPDL23R]
HWY 50 Pink Throw Pillow Covers 18 18 inch, for Couch Sofa Living Room Bed, Decorative Embroidered Square Throw Pillow Case Cushion Cover, Fashion Geometric Floral, Piece
$11.59
[B078NMFQ2S]
ASATEX Crushed SemirSheer White Color Patio Door Window Curtain Panel That's 's 100 Inch Wide and 84 Inch Long Great for Sliding Doors. Extra Large Windows Outdoor Coverings. RIT 100" 84" White
$100.0
Action 2: click [B09KV9XXC9] Observation Instruction:
Find me white item window coverings living room. dining room with color: color13 and size: 84"w 84"1, and price lower 150.00 dollars [Back Search] [< Prev] size
[52x24in] x36in [52x45in]
color
[cactuslop2663] [dandetionlop23981 [degreelop57101 [octopuslop$353]
Blackout Window Curtain Panel White Dandelion Pes Curtain Coverings Rustic Wond
Barn Retro Farmhouse Grommet Top Window Drape for Bedroom Dining Living Room 52
x72in
Price: $35.81 Rating: N.A.
[escriptioo]] [Features] [Reviews] [Buy Now]
Action 3: click [color13]
Observation 3: You have clicked color13.
Action 4: click w84 84"1l Observation 4: You have clicked 84"w 84"1.
Action 5: click [Buy Now]
B.2 ACTOR-SUMMARIZER
Instruction: Find me white item window coverings for living room, dining room with color: color and size: 84"w 84"1, and price lower than 150.00 dollars. [earch]]
Action 1: search I white item window coverings for living room diaing room]
Observation 1:
Find me white item window coverings for living room, dining room with color: color13 and size: 84"w x 84"1, and price lower than 150.00 dollars Back to Search] Search results for keyword "white item window coverings for living room dining room
8

---

Page (Total results: 50)
Next
[B0KKV9XC9]
Blackout Window Curtain Panel White Dandelion Pes Curtain Coverings Rustic Wood
Barn Retro Farmhouse Grommet Top Window Drape for Bedroom Dining Living Room
x72in
$35.8
[09KVTZTN4]
Blackout Window Curtain Panel Coastal Print 1 Pes Curtain Coverings Conch Grommet Top Window Drape for Bedroom Dining Living Room 52x84in
$40.5
[B09KV89NNP]
Blackout Window Curtain Panel Black and White Tulip I Pes Curtain Coverings Flowers Blooming Grommet Top Window Drape for Bedroom Dining Living Room 52x90in
$41.82
The following is in both the instruction and the description
White
Dining Room
Living Room
Thw following is in the instruction, but not in description. Maybe need to check previous page color
size price
Action 2: click [B09KV9XXC)] Observation 2: [Bak Search] [< Prev] Product info of B09KV79XC9; size
[52x24in] [52x36in] [52x45in]
color
[cactuslop2663] [dandelionlop2398] [degreclop5710] [oc$opuslop$353]
Blackout Window Curtain Panel White Dandelion 1 Pes Curtain Coverings Rustic Wood
Barn Retro Farmhouse Grommet Top Window Drape for Bedroom Dining Living Room
x72in
Price: $35.81 Rating: N.A.
Description] [Features [Reviews] [Buy Now]
Action 3: think The item has no color13 option, maybe should get back to search and reformulate the query.] .] Observation 3:
OK.
Action 4: click Back to Search Observation 4: Find me white item window coverings for living room, dining room with color: color and size: 84"w x 84"1, and price lower than 150.00 dollars. [Search]
Action 5: search I white item window coverings for living room dining room color13) Observation 5:
Find me white item window coverings for living room, dining room with color: color13 and size: 84"w x 84"1, and price lower than 150.00 dollars
Back to Search]
Search results for keyword "white item window coverings for living room dining room color13":
Page I (Total results: 50)
[ Next >]
Products that does not match instruction: B09KV79XC9
[B88QQVCKG]]
Animal Triple -Weaved Texture Curtain, Artistic Portrait of White Tiger Wild Nature Predator Watercolor Splashes Window Covering for Living Room, Each Panel 60" w 90" L Black Grey White
$124. [B8SCCZZGG]]
Abstract Triple-Weaved Texture Curtain, Abstract Artwork Consist of Various Shapes and Objects with Vivid Colors Flower Window Covering for Living Room. Each Panel
9

---

60"
90" Multicolor
$124.
[BOSSCY27BV]
Abstract Triple-Weaved Weaved Texture Curtain. Circles Squares Mosaic Pattern Pastel Colors Modern Geometric Tile Illustration Window Covering Multi
Living Room, Each
Panel 60"
$124.1
The following is in both the instruction and the description
white item
window coverings
living room
dining room
color
size
price
The following is in the instruction, but not in description. Maybe need to check previous page
color13
Action 6: click [BOSSQVCKG3] Observation 6: Find me white item window coverings for living room. dining room with color: color13 and size: 84"w 84"1, and price lower than 150.00 dollars [Back Search] [< Prev]
Product info of BORSQVCKG3: size
[84"w x 84"1]
color
[color13]
Animal Triple -Weaved Texture Curtain Artistic Portrait of White Tiger Wild Nature
Predator Watercolor Splashes Window Covering for Living Room, Each Panel 60" W <90" L Black Grey White
Price: $124.1 Rating: N.A.
[escriptiio]] [Features] [Reviews] Buy Now]
Action click [coorr]]] Observation
You have clicked color13
Action click [84"w x 84" Observation 8: You have clicked 84"w x 8
Action 9: click Buy Now
ASH Prompts
We put out our prompt templates used for the SUMMARIZER and ACTOR in ASH method here. Depending the environment and desired behavior,
easily adapt the SUMMARIZER prompt to guide the ACTOR in the right direction for that particular use case.
C.1 SUMMARIZER Prompt for CODE-DAVINCI-002
Action
[SEP]
None
Observation
[SEP]
WebShop
Instruction:
want to find white blackout shades that are 66 inches in width and inches in
height. they need to be easy to install and price lower than 110.00 dollars
[btton]] Search [button_]
Information [SEP]
Let 's think step by step. Based on the observation and action what information will be useful for dumb actor to buy the correct product? need to extract useful
10

---

Neck Tees with information for actor build the correct search keyword.
info Find me white blackout shades with size 66" 66". They need easy install. and price lower than 110.00 dollars. Search]]
Action
[SPP]
search wwomen's tops, tees blouses, light weight. loose fit short sleeve, army
green solid button. large]
Observation
[SEP]
Instruction:
Find me light weight, loose fit women't s tops, tees & blouses with short sleeve with color: army green solid button, and size: large and price lower than 60.00
dollars
button Back to Search [button_1
Page 1 (Total results: 50) I buttonn] Next > [button_]
button]] BO9RWQT4DC [button_]
Women's Cold Shoulder T-Shirt Tops Blouses 137 Crew-Neck Short Sleeve Printing
Strappy Tray Fitting fit Bra Breastfeeding Men't s Fixed Royal Crochet Bathing Casual Clothing Valentines
$19.85
[utton] BO9Т6ЖKIIР IP [button_]
Shirts for Men's Casual Short Sleeve Shirts Summer Button Down Solid Color T- Shirt Tops Beach Loose Comfy Blouse
$3.68
[utton]] BORWRSHQQ [button_]
Women's Print Casual Tops Blouse T-Shirt 435 Short Sleeves Crew-Neck Letter Royal Cowl dot Girls Strapless Sleeveless Fitting Bra Adult Maternity
Asymmetrical breastfeedi
$16.82
button] BORRRRSQSB [button_]
Women's Soft Casual Blouse Print Tops 346 Easter Short Sleeve Shirt ough Breastfeeding 2022 Lucky Animal Cold Fall y90k Leather Cross Swim Tankini
dupes Leggings cro
$17.15
button]] B09T6WXZF4 [button
Shirts for Men's Casual Short Sleeve Button Down Shirt Tops Beach Loose Comfy Blouse
Shirts Summer Solid Color
$1.77
[bttonn] B09QSTV87Q [button_]
Blouses for Women Long/ Short Sleeve Solid Color Shirts Button -Down
Pockets Tops
$12.29 to $14.28
button] B09S3P49JZ [button_]
Loose Printed Sleeveless 266 Women's s V-Neck Tank Shirt Blouse Top Block Boats
Boutique Trendy Trending Dresser Out Strapless Embroidered Racerback Sleeved Short Ladie
$20.84
button B09NNTZCLA [button_] Valentines Day T Shirt for Women, Womens Tops Plus Size Cute Love Heart Graphic Tees Shirts Short Sleeve Tops
$1.21 to $3.21
[ button] B09RWRFI191 [button_] Women's Neck Casual Short Sleeves Tops 931 Stripes Splicing Round Strap Boys' Gym Waist Bodysuits Kinky Suit Tunic Leather Breastfeeding Peplum Sheer Size cor 620.6
]utton BOSSENEWaW [button_]
Casual Easter Printing 280 Women's O-Neck Loose T-Shirt Blouse Tops Short Sleeves Strap Sheer Bikini Workout Cropped Winter Tunic Tube Adults See Bodysuits Teen
Out Serub
$16.22
Information
[SP]]
11

---

Let's think step by step. Based on the observation and action. what information will dumb actor buy the correct product? First, since actor dumb. should choose only about three producls that is most likely to match can discard the all products does not match the instruction for sure have some conflicting attributes). then should hint the dumb actor to search different keyword.
useful
the product and attribute specified by the instruction. remaining products.
info Find me light weight, loose fit women't s tops, tees blouses with short sleeve with color: army green solid button, and size: large, and price lower than 60.00 dollars.
Back to Search]
Search results for keyword "women's s tops. tees & blouses. light weight, loose fit. short sleeve. army green solid button, large":
Page (Total results: 50)
[Next >]
BBRRWQTADC]
Women's Cold Shoulder T-Shirt Tops Blouses 137 Crew-Neck Short Sleeve Printing
Strappy Tray Fitting fit Bra Breastfeeding Men's s Fixed Royal Crochet Bathing Casual Clothing Valentines
$19.85
[BORWRSHHD]]
Women's 's Print Casual Tops Blouse T-Shirt 435 Short Sleeves Crew-Nech Letter Royal Cowl dot Girls Strapless Sleeveless Fitting one Bra Adult Maternity Asymmetrical breastfeedi
$16.82
B09RWRSQSB]
Women's Soft Casual Blouse Print Tops 346 Easter Short Sleeve T Shirt ough Breastfeeding 2022 Lucky dot Animal Cold Fall y90k Leather Cross Swim Tankini dupes Leggings
$17.151
Action
[SEP]
click [078GWRCIJ]
Observation
[SEP]
Instruction:
would like ounce bottle of bright citrus deodorant for sensitive skin price lower than 50.00 dollars button] Back Search [button_]
button] Prev [button_]
scent
[utton assorted seents [button_]
[button
bright citrus [button_] calming lavender [ ginger fresh [button_]
[button
[button
[button]
simply non-scents [button_]
size
[button] travel set (4-pack) [button_ [button] 3 ounce (pack of 1) [button_]
[button] 3-ounce (2-pack) [button_ Bright Citrus Deodorant by Earth Mama Natural and Safe for Sensitive Skin, Pregnancy and Breastfeeding Contains Organic Calendula 3-Ounce
Price: $10.99
Rating: N.A.
[button] Description [button_]
|btton] Features [hutton_]
[button] Reviews [button_]]
[butonn] Buy Now [button_]
Information [SEP]
Let think step step. Based on the observation and action. what information will be useful for dumb actor buy the correct product? First, I need to let the dumb actor know this if this is the correct product. Then, I need filter options that does match the instruction make casier for the dumb actor choose the correct option. Finally. if there is missing information in
12

---

short description. should hint the dumb actor to check the description before buying it info [Find me ounce bottle of bright citrus deodorant for sensitive skin price lower than 50.00 dollars [Bak Search]
k Prev]
Product info of BI78GWRCIJ:
scent
[bright citrus]
size
[3 ounce (pack of 1)]
[3 ounce (pack of 2)]
Bright Citrus Deodorant by Earth Mama Natural and Safe for Sensitive Skin,
Pregnancy and Breastfeeding. Contains Organic Calendula 3-Ounce Price: $10.99
Rating: N.A.
Description Features]
R
Buy Now]]
Action
[SEP]
click Description]
Observation
[SEP]
Instruction:
Find loose fit wide leg. quick drying machine wash men't pants with drawstring
closure clastic waist with color: army green, and size: x-large, and price lower than 40.00 dollars
button] Back Search [button_]
button Prev [button_]
Men Outdoor Recreation Pants Convertible Quick Dry Lightweight Zip Off Outdoor
Fishing Travel Pants Men's Outdoor Casual Blastic Waist Lightweight Water
Resistant Quick Dry Cargo Fishing Hiking Pants Mens Pants Casual Workout
Drawstring Pants with Elastie Waist and Zipper Pockets Mens Pants Casual
Comfortable Workout Pants Drawstring Zipper Pockets Elastic Waist Mens Elastic
Waist Drawstring Workout Gym Joggers 3/4 Pants Summer Casual Sweat Short Pants
with Pockets Men Tactical Pants Waterproof Hiking Fishing Breathable Quick Dry
Cargo Short Pants Size:S US:4 UK:8 UE:34 Waist:64cm////
:ip:86cm/333
:88cm/34.65 Length:8Scm/34.
Size US:6 UK:10 UF:36 Waist:68em/26.77
Hip #ip:900m/355.4
Length 1-990mm35.44- Size US:8 UK:12 UE:38 Waist:72cm/28.33
Hip:9 4cm/37 00nn/7.011 01 01" 9tip:98mm338..m
Length 99t.mm2..22222 Size XXL US:10 UK: 14 UE:40 Waist:76cm/29
Length 94ccm/97000 /40.16" Length :9cem/37.03
" "Size :XLL US:12 UK:16 E:42 Waist:80cm 31.50* Hip:102em
Information [SEP]
Let think step by step. Based on the observation and action. what information will he useful for dumh actor to buy the correct product? First, I need to extract information from the description to let the dumb actor know this if this is the correct product. Those information need to be related to the instruction, and be concise and easy for dumb actor to understand.
info [Find me loose fit, wide leg. quick drying machine wash men 's pants with drawstring closure elastic waist with color: army green. and size: x-large. and price lower than 40.00 dollars Back to Search]
I< Prev]
Deseription of this product
The following is in both the instruction and the description
Quick Dry
elastic waist
drawstring closure
Thw following in the instruction. but not in description. Maybe need check previous page
machine wash
loose fit wide leg
13

---

color size
price]
C.2
1. Instruction: This is the description of the product that the agent is looking for SUMMARIZER Prompt for GPT-3.5-TURBO
You autonomous intelligent summarizer tasked with condensing webpage information by appropriately summarizing removing and adding relevant information. that enables autonomous web navigating agent quickly and successfully buy desired product.
Here's s the information you will have:
Previous Action of Agent: This is the action the agent took previous step that lead to current webpage.
Observation: This is the full representation the webpage agent is currently on, after performing Previous Action, that you have to condense summarize.
The agent that you want to help will be given the following information: 1. Instruction: This is the description of the product that the agent is looking for
2. Observation i: This is the condensed and relevant representation of the webpage that you provided after the agent performed Action i-l.
3. Action i: This is the action the agent took in step i. after seeing Observation provided by you.
Instruction. followed by a series of Observation and Action will be given to the agent to decide next optimal action.
The agent you want to help can perform the following actions:
1. search]<query>
2. click[<button_name:
3. think[<texi>]
To be successful, it is very important for
follow the following rules: 1. First. reason out how you want to condense the Observation for the current state and what information you need to add, if any.
2. Next, present one Condensed Observation in the following form: info ConodeCndddseedd Observation>]
3. Do not remove navigating buttons since the agent must be able to go back and forth between pages.
4. When there is long list of options products in the webpage, try retain only couple of them in the Condensed Observation based on the Instruction and Previous Action of Agent.
5. If "Invalid Action" is found in the Observation add hint within Cnfol<Cnndensed Observation >] help the agent perform the correct action. Always add click) after the attribute Condensed Observation.
like color, size, etc. in the
You are given solved examples help you understand your task.
Instruction:
WebShop
Instruction:
want to find white blackout shades that are 66 inches in width and inches in height. they need to be easy to install and price lower than 110.00 dollars [btton]] Search [button_]
### Previous Action Agent:
None
*#и Observation:
WebShop
Instruction:
I want to find white blackout shades that are 66 inches in width and 66 inches in height. they need to be easy to install and price lower than 110.00 dollars button] ] Search [button_]
14

---

Condensed Observation:
First. need remove the instruction since it given the agent already. Finally will retain the Search button so that no action is left out.
info [[ [[Search]]
### Instruction:
WebShop
Instruction:
Find me light weight, loose fit women's s tops, lees & blouses with short sleeve with color: army green solid button, and size: large, and price lower than 60.00
dollars
### Previous Action of Agent: search [women's s tops, tees & blouses, light weight, loose fit, short sleeve. army green solid button, large]
### Observation:
WebShop
Instruction:
Find me light weight, loose fit women's s tops, lees & blouses with short sleeve with color: army green solid button, and size: large and price lower than 60.00
dollars
[uttonn] Back Search [button_]]
Page 1 (Total results: 50)
button Next [button_
button] BORROOTAD:: [button_]
Women's s Cold Shoulder T-Shirt Tops Blouses 137 Crew-Neck Short Sleeve Printing
Strappy Tray Fitting fit Bra Breastfeeding Men Fixed Royal Crochet Bathing Casual Clothing Valentines
$19.85
[btton]] B09Т6XKI1P [button_]
Shirts for Men,Men's Casual Short Sleeve Shirts Summer Button Down Solid Color T- Shirt Tops Beach Loose Comfy Blouse
$3.68
[utton]] BORWRSHOQ [button_]
Women's Print Casual Tops Blouse T-Shirt Short Sleeves Crew-Neck Letter Royal Cowl dot Girls Strapless Sleeveless Fitting one Bra Adult Maternity
Asymmetrical breastfeedi
$16.82
button] BOARWRSQSB [button
Women's Soft Casual Blouse Print Tops 346 Easter Short Sleeve Shirt ough Breastfeeding 2022 Lucky dot Animal Cold Fall y90k Leather Cross Swim Tankini
dupes Leggings ero
$17.15
button] B09T6WXZF4 [button_]
Shirts for Men.Ment Casual Short Sleeve Button Down Shirt Tops Beach Loose Comfy Blouse
Shirts Summer Solid Color T-
$1.77
[utton]] B09QSTV87Q [button_] ]
Blouses for Women Long/ Short Sleeve Solid Color Shirts Button- -Down Neck Tees with Pockets Tops
$12.29 to $14.28
I button 1 B09S3P49IZ [button_]
Loose Printed Sleeveless 266 Women's V-Neck Tank Shirt Blouse Top Block Boats
Boutique Trendy Trending Dresser Out Strapless Embroidered Racerback Sleeved Short Ladie
$20.84
[butonn] BO9NNTZCLA [button_] Valentines Day T Shirt for Women, Womens Tops Plus Size Cute Love Heart Graphic Tees Shirts Short Sleeve Tops
$1.21 to $3.21
button] B09RWRFH91 [button_]
Women' Neck Casual Short Sleeves Tops 931 Stripes Splicing Round Strap Boys' Gym Waist Bodysuits Kinky Suit Tunic Leather Breastfeeding Peplum Sheer Size cor 620.6
button] B09S3NFW4W [button_]
Casual Easter Printing 280 Women's O-Neck Loose S-Srirt Blouse Tops Short Sleeves Strap Sheer Bikini Workout Cropped Winter Tunic Tube Adults See Bodysuits Teen
Out Scrub
15

---

$16.22
Condensed Observation:
First need to remove the instruction since it is given the agent already. Then the
will choose only the three products closest to the description Instruction. can discard the remaining products. If all products are far from the description in instruction then should hint the actor to go back and reformulate search query. Finally, will retain all navigating buttons so that no action is left out.
info Back to Search]
Page (Total results: 50)
[Next- >]
[B0RRQT4DD]]
Women's Cold Shoulder T-Shirt Tops Blouses 137 Crew-Neck Short Sleeve Printing
Strappy Tray Fitting fit Bra Breastfeeding Men s Fixed Royal Crochet Bathing Casual Clothing Valentines
$19.85 85
BOARWRSHQD]
Women's 's Print Casual Tops Blouse T-Shirt 435 Short Sleeves Crew-Neck Letter Royal Cowl dot Girls Strapless Sleeveless Fitting one Bra Adult Maternity Asymmetrical breastfeedi
$16.82
[B9RWRSQSB]
Women's 's Soft Casual Blouse Print Tops 346 Easter Short Sleeve T Shirt ough Breastfeeding 2022 Lucky dot Animal Cold Fall y90k Leather Cross Swim Tankini dupes Leggings ero
$17.15]
### Instruction:
WebShop
Instruction:
Find light weight. loose fit women's tops, & blouses with short sleeve with color: army green solid button, and size: large and price lower than 60.00 dollars
### Previous Action Agent: search women's s tops, tees blouses, light weight, loose short sleeve, army green solid button. large]
Observation:
button]] Back Search [button_]]
Page (Total results: 50)
[button]] | Next [button_]
button BORRWOTADC [button_]
Women's Cold Shoulder T-Shirt Tops Blouses 137 Crew-Neck Short Sleeve Printing
Strappy Tray Fitting fit Bra Breastfeeding Men Fixed Royal Crochet Bathing Casual Clothing Valentines
$19.85
[utton]] 109T6XK11P [button_]
Shirts for Men, Men,Men's Casual Short Sleeve Shirts Summer Button Down Solid Color T- Shirt Tops Beach Loose Comfy Blouse
$3.68
button] BORWRSHQQ [button_]
Women's Print Casual Tops Blouse T-Shirt 435 Short Sleeves Crew-Neck Letter Royal Cowl dot Girls Strapless Sleeveless Fitting one Bra Adult Maternity
Asymmetrical breastfeedi
$16.82
Invalid Action.
### Condensed Observation:
First, Invalid Action is seen and the previous action was search []. but there is no search button in the webpage. Henee, should add hint that there is search button and the agent needs go search page. Then, will retain the two products that are elose the deseription in the Instruction. can discard the remaining products. Finally. will retain all navigating buttons that action is left info [[ Back Search]
Page I (Total results: 50)
Next
BORWQT4DC]
16

---

Women's Cold Shoulder T-Shirt Tops Blouses 137 Crew-Neck Short Sleeve Printing Strappy Tray Fitting fit Bra Breastfeeding Men's Fixed Royal Crochet Bathing Casual Clothing Valentines
$19.85
[BORWWSHIQD]
Women's Print Casual Tops Blouse T-Shirt 435 Short Sleeves Crew-Nec Letter Royal Cowl dot Girls Strapless Sleeveless Fitting one Bra Adult Maternity Asymmetrical breastfeedi
$16.82
Hint: No Search button found on this webpage, Go to search page.
### Instruction:
WebShop
Instruction:
would like ounce bottle of bright citrus deodorant for sensitive skin, and price lower than 50.00 dollars
### Previous Action of Agent:
click (B78GGRRI)
### Observation:
WebShop
Instruction:
would like 3 ounce bottle of bright citrus deodorant for sensitive skin, and price lower than 50.00 dollars I Back Search [button_]
button]
Prev [button_]
seent
[button] assorted scents [button_] bright citrus [ calming lavender [button_] ginger fresh [button_]
[button
[button]
[button]
[button]
simply non-scents [button_]
size
[button travel set (4-pack) [button_] [button] 3 ounce (pack of 1) [button_]
[button 3-ounce (2-pack) b
Bright Citrus Deodorant by Earth Mana Natural and Safe for Sensitive Skin. Pregnancy and Breastfeeding, Contains Organic Calendula 3-Ounce
Price: $10.99
Rating: N.A.
[button] Description [button_]
[button] Features [button_] [button] | Reviews [button_]
[button] Buy Now [button_
### Condensed Observation:
First need to remove the instruction since it is given to the agent already. Then need to filter out options that does not match the instruction to make it casier for the agent to choose the correct option. If there is missing information in the short description, I should hint the agent to check the description before buying it. If there are missing options. I should hint the agent this might not be the correct product it searched for. Finally, will retain all navigating buttons so that no action is left out. [ to Search]
j Prev]
Product info of B07SGWRCIJ;
scent (to click)
[bright citrus]
size (to click)
[3 ounce (pack of 1)]
13 ounce (pack of 2)
Bright Citrus Deodorant by Earth Mama I Natural and Safe for Sensitive Skin, Pregnancy and Breastfeeding, Contains Organic Calendula 3-Ounee Price: $10.99
Rating: N.A.
Description] [eatures]
Reviews] Buy Now]]
17

---

C.3 ACTOR Prompt for CODE-DAVINCI-002
Instruction: Find me white blackout shades with size 66" 66". They need to be easy to install, and price lower than 110.00 dollars. [Search]
Action I: search166 inches in width and 66 inches in height white shades Observation I:
Find me white blackout shades with size 66" 66". They need to be easy install and price lower than 110.00 dollars.
BBak Search]
Search results for keyword "66 inches in width and 66 inches in height white shades
":
Page (Total results: 50)
[Next
Products clicked before: None
BB8QGG4DYY]]
Maxxmore Tiffany Pendant Light L-gight Width Mission Style Stained Glass Hanging
Lamp Kitchen Island Dining Room Foyer
$164.0
(B07L64LC6Y)
Calyx Interiors Inch Snow White Cordless Faux Wood Blind with Royal Valance, 46.5-Inch Width x 48-Inch Height
$100.0
[08PP4YM9D]
Hmlinktt Couch Gray Convertible Modern Faux Leather Futon Sofa Bed with 2 Cup Holders and Armrest Folding Recliner, for Living/ Office Room ( Black)
$435.0
Action 2: click [B07L64LC6Y]
Observation 2: Find me a white blackout shades with size 66" x 66". They need to be casy to install and price lower than 110.00 dollars. Back to Search]
k< Prev]
Product info of B07L6ALC6Y: size
[19-inch width 60-inch height] 199.125innhh width x 60-inch height]
(21.5-inch width x 60-inch height]
Calyx Interiors Inch Snow White Cordless 2" Faux Wood Blind with Royal Valance, 46.5-Inch Width x 48-Inch Height
Price: $100.0
Rating: N.A.
[Description]
Features]
[Reviews] [Buy Now]
Action think The item has 66x66 option, mayhe I should get back search and reformulate the query.] Observation OK.
Action click Back to Search Observation Find me a white blackout shades with size 66" x 66". They need easy install and price lower than 110.00 dollars. [Search]
Action 5: search[66 Observation 5: Find me a white blackout shades with size 66" x 66". They need to be easy to install and price lower than 110.00 dollars.
blackout shades
[Back to Search]
Search results for keyword "66 66 blackout s:ades": Page 1 (Total results: 50):
[Next >]
Products clicked hefore: B07164L.C6Y [B09TXXCWP]]
18

---

MiLin Blinds Windows Cordless Blackout Cellular Shades, Blinds Shades Room Darkening Honeycomb Shades Blinds Heat Insulation Protection for Home Kitchen Bedroom Office While Wx x 36"
$78 $78.49
[B082GGGM2Y]
AWAY Portable Blackout Curtain with Window Suction Cups (66" Long x 51" Wide) Temporary Room Darkening Shades for Baby Nursery, Bedroom; Dorm Room Window
Cover for Travel
$42.99
[B09GP34TDR]
Ejiawj Painted Total Blackout Curtains for Bedroom Plant Comics Farm House Curtains 66 x 72 inch Grommet Room Blackout Curtains Heat Insulation Noise Reduction and Blackout Curtains Suitable for Livi
$100.0
Action 6: click [80STXXCWP]] Observation 6:
Find me a white blackout shades with size 66" x 66". They need to be casy to install and price lower than 110.00 dollars. [Back Search]
Product info of B09TKXCWP7: k Prev]
size
[66'w 56"h]
[66-w x 60"h]
[66'w 66"h]
color
cordless bottom up-blackout-crremy
[cordless bottom laoblackout-grayy
[cordless bottom up-blackout-white.
MiLin Blinds for Windows Cordless Blackout Cellular Shades. Blinds Shades Room Darkening Honeycomb Shades Blinds Heat Insulation UV Protection for Home Kitchen Bedroom Office White 28" W x 36" H
Price: $78.49
Rating: N.A.
Description] Features [ [Buy Now
Action 7: think Foor white blackout shades that are inches width and inches height. item has options '66"v "w x 66"h" and 'cordless bottom up up-blackout- white so should buy this item.] Observation 7:
OK.
Action 8: click [66"w x 66"h] Observation 8: You have clicked 66"w x h
Action 9: click coorlesss bottom up-blackout-ve white] Observation 9: You have clicked cardless bottom up-blackout - white.
Action 10: click Buy Now]
Instruction: Find me ounce bottle of bright citrus deodorant for sensitive skin. and price lower than 50.00 dollars [
Action 1: search[3 ounce bright citrus deodorant sensitive skin Observation 1: Find me
bottle of bright citrus deodorant for sensitive skin and price lower than 50.00 dollars
[Bak Search
Search results for keyword "3 bright citrus deodorant sensitive skin": Page (Total results: 50)
[Next
19

---

Products clicked before: None
[B07&GWRCIJ]
Bright Citrus Deodorant by Barth Mana Natural and Safe for Sensitive Skin Pregnancy and Breastfceding, Contains Organic Calendula 3-Ounce
$10.99
[B078GTKVXY]
Ginger Fresh Deodorant by Earth Mama Natural and Safe for Sensitive Skin, Pregnancy and Breastfeeding, Contains Organic Calendula 3-Ounce
$10.99
[B0KKBVI4XN]
Barrel and Oak Aluminum-Fret Free Deodorant, Deodorant for Men, Essential Oil-Based Scent, 24-Hour Odor Protection Cedar & Patchouli Blend, Gentle on Sensitive Skin (Mountain Sage, 2.7 oz. 2-Pack)
$15.95
Action 2: click [807&GWRCIJ] Observation 2:
Find me a 3 ounce bottle of bright citrus deodorant for sensitive skin, and price lower than 50.00 dollars [Back to Search]
k< Prev]
Product info of BOTSGWRCIJ; scent
[bright citrus]
size
13 ounce (pack of 1)1
13 ounce (pack of 2)
Bright Citrus Deodorant by Earth Mama Natural and Safe for Sensitive Skin, Pregnancy and Breastfeeding. Contains Organic Calendula 3-Ounce Price: $10.99
Rating: N.A.
[Description] [Features]
] [Buy Now
Action 3: click bright citrus] Observation
You have clicked bright citrus.
Action 4: click [3 ounce (pack of 1)]
Observation 4:
have clicked ounce (pack 1).
Action 5: click Buy Now]
C.4 ACTOR Prompt for GPT-3.5-TURBO
You autonomous intelligent agent tasked with navigating shopping website called "WehShop" buy desired product. These tasks will be accomplished through the use of specific actions you can issue.
Here's s the information you' '1 have:
Instruction: This is the description of the product you are looking for. Observation This
simplified representation of webpage you face after performing Action i-1, providing key information
buttons you can click on. 3. Action i: This is the action you took step i, after seeing Observation Instruction will found at the beginning of the task, followed by a series of Observation and Action. (I
You can perform the following actions:
I. search|<query>1 - search for products described by <query
2. click[<button name >] click on button present in webpage where <button name is the name of the button given as [button name] in Observation.
3. [hink[<text>] - think and reason about what needs to be done given the history of observations and actions. Your next action will be based on this thought.
To be successful, it is very important to follow the following rules:
20

---

Issue Action in correct format. ensuring is supported by the Observation. and issue only one Action at time.
not repeat Action like clicking "Invalid Action" is found in the Observation the Action performed is not supported by the webpage. Try a different Action that is supported by the webpage or navigate to the webpage that supports that Action. not perform [hiek[<text>] Action Observation.
same product or searching the same query.
"Thought Through" is present the
Do not always look for exact string matches between Instruction and the product description in the webpage. Look if the meaning of the product description matches the Instruction. Follow Hint in Observation carefully.
You are given few solved examples to help you understand your task.
### Instruction:
WebShop
Instruction:
want to find white blackout shades that are 66 inches in widhh and 66 inches in height. they need to be easy to install and price lower than 110.00 dollars |earch]
### Observation 1:
WebShop
Instruction:
want to find white blackout shades that are 66 inches in width and 66 inches in height. they need to be easy to install and price lower than 110.00 dollars |earch
### Action 1:
search inches width and 66 inches height white shades]
### Observation 2:
Back Search]
Page (Total results: 50)
Next
B08QGG4DY7]
Maxxmore Tiffany Pendant Light 2-Light Width Mission Style Stained Glass Hanging Lamp for Kitchen Island Dining Room Foyer
$164.0
[B07L64LC6Y]
Calyx Interiors Inch Snow White Cordless Faux Wood Blind with Royal Valance. 46c5-Inch Width x 48-lnch Height
$100.0
[08P24YMMD]]
Hmlinktt Couch Gray Convertible Modern Faux Leather Futon Sofa Bed with Cup Holders and Armrest Folding Recliner, for Living/ Office Room ( Black)
$435.0
### Action 2:
click (B07L64LC6Y)
### Observation 3:
I Back to Search
[< Prev]
size [19-inch width 60-inch 1299. width 60-inch height 1/215 inch width 60-inch height]
Calyx Interiors Inch Snow White Cordless 2" Faux Wood Blind with Royal Valance
Inch-Inch Width x 48-Inch Height Price: $100.0
Rating: N.A.
Description]
Features ]
|Buy Now]
### Action 3:
think [Te item has 66x66 option, maybe should get back search and reformulate the query.]
21

---

Observation Thought Through.
Action click Baak Search]
*** Observation 5:
WebShop
Instruction:
want to find white blackout shades that are 66 inches in width and inches in height. they need to be easy to install and price lower than 110.00 dollars Search]
### Action 5: search [66 x 66 blackout shades]
###
Observation 6: Back to Search]
Page 1 (Total results: 50)
[Next >]
[B99KXXCP7]]
MiLin Blinds for Windows Cordless Blackout Cellular Shades. Blinds & Shades Room
Darkening Honeycomb Shades Blinds Heat Insulation UV Protection for Home Kitchen
Bedroom Office White 28" W x 36" H
$78.49
[B82GGGG2YY
SUN AWAY Portable Blackout Curtain with Window Suction Cups (66" Long 51" Wide Temporary Room Darkening Shades for Baby Nursery, Bedroom, Dorm Room Window
Cover for Travel
$42.99
[B09GP34TDR]
Ejiawj Painted Total Blackout Curtains for Bedroom Plant Comics Farm House Curtains 66 72 inch Grommet Room Blackout Curtains Heat Insulation Noise Reduction and Blackout Curtains Suitable for Livi
$100.0
Action click [В097КXCWP]]
## Observation 7:
[Back Search]
[ Prev] size
[66"w x 36"h][66" 60"h]166"w x [6"h][custon" size
color
coordless bottom up-blackout-ereamy I[ cordless bottom up-blackout-gray -gray I[ cordless bottom ul-bkaakout manekout-white
MiLin Blinds for Windows Cordless Blackout Cellular Shades, Blinds Shades Room Darkening Honeycomb Shades Blinds Heat Insulation UV Protection for Home Kitchen Bedroom Office White 28" W x 36" H
Price: $78.49
Rating: N.A.
[eseriptioo]]
Features] ]
|Buy Now]
##и Action 7:
think [For white blackout shades that are 66 inches in width and 66 inches in height. the item has options '66"w 66"h and 'cordless bottom up-blackout-whirg ', should buy this item.]
### Observation 8: Thought Through.
*## Action 8: click166"w 66"h]
### Observation
You have clicked 66"w h
22

---

Action
click cordless bottom ap-blackout-whig
Observation
have clicked cordless bottom up-blackout white.
### Action 10: click [Buy Now]
### Instruction:
WebShop
Instruction:
would like ounce bottle of bright citrus dcodorant for sensitive skin, and price lower than 50.00 dollars [
### Observation 1:
WebShop
Instruction:
would like ounce bottle of bright citrus deodorant for sensitive skin, and price lower than 50.00 dollars ]earchh]
### Action I: search ounce bright citrus deodorant sensitive skin
### Observation 2:
[Back Search]
Page (Total results: 50)
[Next
[B078GWRCIJ]
Bright Citrus Deodorant by Earth Mama Natural and Safe for Sensitive Skin, Pregnancy and Breastfeeding Contains Organic Calendula 3-Ounce
$10.99
[B07&GTKVXY]
Ginger Fresh Deodorant by Earth Mama Natural and Safe for Sensitive Skin, Pregnancy and Breastfeeding. Contains Organic Calendula 3-Ounce
$10.99
[BOSKBVJ4XN]
Barrel and Aluminum-Fres Deodorant, Deodorant for Men, Essential Oil-Based Scent. 24-Hour Odor Protection Cedar & Patchouli Blend. Gentle on Sensitive Skin (Mountain Sage, 2.7 oz, 2-Pack)
$15.95
### Action 2:
click [B07SGWRCIJ]
*## Observation 3:
[Bak to Search]
[< Prev]
scent
[assorted scents briight citrus I| calming lavender [ ginger fresh ][ simply non-scents] size
[travel set (4-pack) I13 ounce (pack of 1) ounce (2-pack) l
Bright Citrus Deodorant by Earth Mana I Natural and Safe for Sensitive Skin, Pregnancy and Breastfeeding, Contains Organic Calendula 3-Ounce
Price: $10.99
Rating: N.A.
[Description] [Features]
] [Bu Now1
### Action 3: click [bright citrus]
*## Observation 4: You have clicked bright citrus.
23

---

Action
elick13 ounce (pack I)1
### Observation 5:
You have clicked ounce (pack 1).
##*
Action 5: click Buy Now]
SUMMARIZER Output Example
Here we show example, where the SUMMARIZER eliminates irrelevant information by picking the top three relevant products in the search result.
observation (input SUMMARIZER)
Instruction:
Find me loose
wide leg. quick drying machine wash men't pants with drawstring closure, elastic waist with color: army green. and size: x-large. price lower than 40.00 dollars
[button] Back to Search [button_]
Page (Total results: 50)
[button] Next > [button_]
[btton]] B04Q6KK8WC [button_]
Classic Fit Ripstop Cargo Shorts Big and Tall Outdoor Casual Quick Dry Hiking Tactical Workout Short with Pockets $18.29
Mens
[utton] B09C39317W [button_]
HUANLE Men Fitness Sports Yoga Pants Men's Compression Pants Quick Dry Capri Athletic Leggings Workout Tights
$11.99 to $12.79
I button B09RB3HАKF [button_]
Women's Loose Fit Elastic Waist Drawstring Lace Shorts Plus Size Summer Beach Casual Yoga Lounge Shorts Pants S-5XL.
$9.99
button] B09RZK8742 [button_]
Merthy Mens Cotton Linen Drawstring Pants Casual Classic Fit 11 Inch Inseam Elastic Waist Shorts Summer Beach Yoga Short
$6.14
[button] B08ZRT9NIT [button_]
Mordenmiss Women's Baggy Drawstring Pants Wide Leg Pants Casual Elastic Waist Trousers
$25.99 to $33.99
button] B09SZFTFJ4 [
Merthy Linen Casual Classic Fit Shorts for Mens Drawstring Summer Beach Short
Elastic Waist Sleep Pajama Pants with Pockets
$4.88
button] B09SHTBSTG [button_]
Merthy Mens Lounge Shorts Drawstring Elastic Waist Tactical Workout Cargo Short Loose Fit Gym Fishing Hiking Palazzo Pants
$4.99
button] B094975Y71 [button_]
Wide Leg Pants for Women Elastic Waist Casual Capris Crop Pull Pants Dandelion Harem Trousers Cropped Pants with Pockets
$5,99 $16.88
button]] B09T3LTITN [button_]
Merthy Summer Casual Rompers for Women Deep V Neck Spaghetti Strap Drawstring Waist Terry Jumpsuits Overalls with Pockets $11.99
[utton]] B09R9ZV4VY [button_]
melupa Summer Pants for Women Casual Pockets Cotton Linen Wide Log Trousers Drawstring Elastic Waist Jogging Yoga Sweatpants
$14.99
Summarizer output
Find me loose fit wide leg, quick drying machine wash men s pants with drawstring closure. clastic waist with color: army green, and size: x-large, and price lower than 40.00 dollars
24

---

BBak Search
Search results for keyword "loose fit wide leg quick drying machine men pants drawstring closure elastic waist army green x-large Page (Total results: 50)
[Next
[B090KKKWC]]
Mens Classic Fit Ripstop Cargo Shorts Big and Tall Outdoor Casual Quick Dry Hiking Tactical Workout Short with Pockets
$18.29
B09R2K8742]
Merthy Mens Cotton Linen Drawstring Pants Casual Classic Fit 11 Inch Inseam Elastic Waist Shorts Summer Beach Yoga Short
$6.14
B09SZFTFJ4]
Merthy Linen Casual Classic Fit Shorts for Mens Drawstring Summer Beach Short Elastic Waist Sleep Pajama Pants with Packets
$4. 8888
E SUMMARIZER Output Analysis
E.1 SUMMARIZER Failure Example 1
The SUMMARIZER fail when the raw observation too long. simply repeats many of the options without effective summarization.
Raw Observation:
Instruction:
Find me easy care moisture wicking. machine wash men' 's pants with polyester cotton with color: black, and size: 29w x 301, and price lower than 50.00 dollars button] Back to Search [button_]
button]] < Prev [button_]
color
buttonn]
front [button_] navy-pleated front [button_] brown-pleated front [button_]
olive-plest
[button
[button
[button]
stone -pleated front [button
[button
black -pleated front [button_]
[button]
khaki - -fat front [button_]
[button]
-flak front [button_
[button]
khaki -peeatd front [button charcoal-flat front [button_] white-plest
[button]
[button
front [bttoon_] black skinny fit front [button_] pavy-flat front [
[button]
[button
[button
olive-fla front [button_] white flat front [button_]
[button
size
[button]
28w 281
button
[button
28w
301
[button
[button
28w
321
[button
/buttn
[bttion_
[button
29w
291
[utton
bbuton
29w
301
[uttoon_
[button
29w
321
[button
[button
291
[button
[button
301
[button
[button
30w
321
button
[button
30w
341
[bttoon
[button
31w
291
button
[button
31w
301
button
[buttoo
31w X 321
[button
[button
32w 281
button
[button
32w x 321
[butlon
[button
32w 341
[utton
[button
33w x 281
[button
[button]
33w 301
button
[button
33w 321
|buttnn_
[buttoo]
34w 301
button
25

---

[button
321
[button
[button
36w
301
[button
[button
321
[button..
[button
36w
361
[button
[button
301
[button
[button
38w
321
[button
[button
40w
301
[button
[button
40w
321
button
[button
40w
341
button
[button
42w
301
[button
[button
44w
301
[button
[button
44w
321
[button
[button
44w
341
butttnn
[button
46w
301
button
[button
46w
321
button
[button
48w
301
button
[button
48w
321
button
[button
48w
361
button
[button
50w
301
button
[button
50w
321
[button
[button
52w
301
[button
[button
52w
321
[button
[button
54w
301
[utton
[button
54w
321
[button
[button
56w
301
[button
[button
56w
321
buttonn_
[button]
58w
301
button
[button
58w
321
[button_
[button 60w
301
[button_
[button 60w
321
[buttoo_
AKA Men's Wrinkle Free Cotton Twill Traditional Fit Slacks Chino
Straight Legs
Casual Pants
Price:
$47.99
Rating: N.A.
[button] Deseription [button_|
[button] Features [hutton_]
button Reviews [button_
[butonn] Buy Now [button_]
Summarizer output
me easy care, moisture wicking. machine wash men 's pants with polyester cotton with color: black, and size: 29w x 301, and price lower than 50.00 dollars
Back Search]
k< Prev]
Product info of BOIN45Q695:
color
[black pleated front]
size
[29w x 301]
[29w x 321]
[30w
301
[30w 321
[30w 341
3 1lw
301
[3ww
321
[32w
301
[32w 321
[32w 341
[33w 301
[33w 321
[34w 301
[34w 321
[36w 301
[36w 321]
[36w 3611
[38w 301]
[38w 321]
[40w 301]
26

---

140w 3
140w 3411
142w 301
44w X 301
44w 321
144 X 341
46w X 301
46w X 321
48w X 301
48w X 321
48w X 361
50W X 301
[50w X 321
[52w X 301
[52w X 321
[54w X 301
[54W 321
[56w X 301
[56w 3211
[58w 301]
[58w 3211
[60w 301]
[60w 3
AKA Men's Wrinkle Free Cotton Twill Traditional Fit Slacks Ch
Expected Outpur
Find easy moisture wicking. machine wash men't 's pants with polyester cotton with color: black. and size: 29w 301, and price lower than 50.00 dollars
[Back Search]
k< Prev]
Product info of B0IN45QG95:
color
front]
size
129w 3011
AKA Men's s Wrinkle Free Cotton Twill Traditional Slacks Chino Straight -Lgs Casual Pants
Price: $28.61
$47.99
Rating: N.A.
[Dscriitiion] [eatures]
Reviews] |Buy Now]
E.2 SUMMARIZER Failure Example 2
SUMMARIZER fails because of hallucination. It hallucinates an non-existing size option '[34w x 341]* which is required by the instruction.
observation (input SUMMARIZER)
Instruction:
Find me rise straight leg slim fit. machine wash men jeans with imported
zipper. polyester spandex with color: jet set blue and size: 34w x 341, and price lower than 90.00 dollars button] Back Search [button_]]
button
Prev [button_]
size
[utton
25w x 321 [button_]
[button
30 [bttton_
[button
30w 381 [button_]
[button
32 [button
[button
32w x 301
[button.
[button] 32w 341
button
[button
32w 361
[button_
[button]
32w 381
[butoon
27

---

[button
[button__
[button
33w 301
bbttonn_
[button
33w 381
[button_
[button
[button.
[button
301
[button
[button
34w 361
[button
[button 34w 381
[button.
[button
35w x 361
[button_
[button
36 [button_
[button
36w 301 [button_]
[button 38 [button..
[button 40w x 321 [button_]
[button] 40w 341 [button.
[button] 44w 361 [button_]
ARIAT Men s M4 Low Rise Stretch Shoreway Stackable Straight Leg Jean
Price: $79.99 to $166.75
Rating: N.A.
[button] Deseription [
button Features [button_]
button Reviews [button_]
buutonn] Buy Now [button_]
Summarizer output
Find me low rise, straight leg, slim fit machine wash men's jeans with imported zipper. polyester spandex with color: jet set blue, and size: 34w 341, and price lower than 90.00 dollars
Back Search]
K< Prev]
Product info of B084G9PT3P:
size
[34w 341]
ARIAT Men's s M4 Low Risc Stretch Shoreway Stackable Straight Leg Jean Price: $79.99 $166.75
Rating: N.A.
[Description]
[Features]
R
[Buy Now]
F Environment Setup Code
env WeeAgennTTextEnv From WebShop repo observation_ _addeattiooommd===-ee mode= 'textrrcch', render=False.
num_products=None. # full product space
for seed in (501. 1000):
random. seed seed)
session "" join (random.. choices ( (string ascii lowercase. k=10)) (ob, _) env reset (esssion=ssssion) test LLM agent on this session
28


Reviews:
[{"review_id": "6mSMDcNCnK", "reviewer": "Reviewer_NBxP", "summary": "", "questions": "Have you tested on other LLMs?\nHave you tested modify the prompts in baseline methods to add sth equivalent to the summarizer and test its performances?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thorough reviews and valuable feedback. We understand the concern of testing only two older LLMs in the paper (the result for GPT-3 text-davinci-002 is provided in Appendix B), and have run our prompting method ASH on a more recent and chat-based LLM: ChatGPT (gpt-3.5-turbo). We choose this LLM to balance the strength and cost of running the experiment. We also chose the 4K token limit variant to test generalization to smaller token limit LLMs. While this initial analysis was run on a smaller product space with 1000 products and using 100 test samples, we will adopt the same sophisticated settings as the main results in the paper, and add it to the revised version of the paper along with the respective error analysis.\n\nAs shown in the table below, ASH surpasses ReAct’s performance on ChatGPT, mirroring the same trend as when we were using the relatively weaker Codex models or GPT3. While the major merit of ASH in this setting has been its ability to condense observation allowing longer trajectories on the smaller token limit models, its ability to help agent recover from errors and align to the optimal behavior still holds as we observed on few sample test trajectories. The pronounced performance improvement with ASH can be attributed to the nature of ChatGPT agent: the agent attempts to buy the exact product and does not frequently settle with suboptimal products, compared to the older LLMs. We will also release the code and prompts for the chatgpt experiments in the same github repository as our main code.\n\nThe main advantage of the proposed method is that the SUMMARIZER is orthogonal to the development of better ACTOR prompts, which the existing works have majorly focused on. Infact, our main method uses ReAct based ACTOR prompting conjugated with the SUMMARIZER. We have also demonstrated using a Act based ACTOR prompting and added SUMMARIZER in the loop. In future, we expect that more sophisticated ACTOR and SUMMARIZER prompts can be individually developed and finally combined to achieve superlative performances.\n\n| Model                                                             | **ASH**  Avg Score | **ASH** Success %     | **ReAct**  Avg Score | **ReAct** Success % |\n| ----------------------------------------------------------------- | --------- | --------- | --------- | --------- |\n| Codex (code-davinci-002) on 50 samples                            | 60.9      | 23.1      | 50.2 | 15.3 |\n| GPT3 (text-davinci-002) on 50 samples                             | 62.6      | 28.8      | 61.3 | 23.1 |\n| ChatGPT (gpt-3.5-turbo) on 100 samples | 59.9      | 46.0        | 17.8 | 10.0 |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The proposed method is simple yet effective. The main contribution is a summaizer which can help remove noise. This partially mitigate the error propagation in long chains when using LLMs.", "reasons_to_reject": "Only one LLM is tested here. It is not sure if the proposed method can be generalized to other LLMs.", "questions_to_authors": "Have you tested on other LLMs?\nHave you tested modify the prompts in baseline methods to add sth equivalent to the summarizer and test its performances?", "paper_topic_and_main_contributions": "This presents a simple approach to improve the performance of large language models on interactive decision making. It is tested on a web navigation dataset and beats sota.\n", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "JuLIEn6R68", "reviewer": "Reviewer_g69r", "summary": "", "questions": "Is it possible to test the proposed method on the text-davinci-003-based model to verify the effectiveness of the proposed method?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thorough reviews and valuable feedback. We understand the concern of testing only two older LLMs in the paper (the result for GPT-3 is provided in Appendix B), and have run our prompting method ASH on another new, chat-based LLM: ChatGPT (gpt-3.5-turbo). We choose this LLM to balance the strength and cost of running the experiment. We also chose the 4K token limit variant to test generalization to smaller token limit LLMs. While this initial analysis was run on a smaller product space with 1000 products and using 100 test samples, we will adopt the same sophisticated settings as the main results in the paper, and add it to the revised version of the paper along with the respective error analysis.\n\nAs shown in the table below, ASH surpasses ReAct’s performance on ChatGPT, mirroring the same trend as when we were using the relatively weaker Codex models or GPT3. While the major merit of ASH in this setting has been its ability to condense observation allowing longer trajectories on the smaller token limit models, its ability to help agent recover from errors and align to the optimal behavior still holds as observed on few sample test trajectories. The pronounced performance improvement with ASH can be attributed to the nature of ChatGPT agent: the agent attempts to buy the exact product and does not frequently settle with suboptimal products, compared to the older LLMs. In fact when we provide examples of successful trajectories this behaviour is enhanced, as seen by the higher average score when ACTOR is zero-shot prompted. We will also release the code and prompts for the chatgpt experiments in the same github repository as our main code.\n\nOne important ablation study that we wish to add to our revised paper is varying the numbers of in-context examples in the ACTOR prompt to see how that affects the relative performance of ASH vs ReAct. While we look forward to add an in-detail analysis on this in the revised version, we report the performance on two points now (current version that uses a 2-shot ACTOR and a zero-shot ACTOR) here. in the and found that when we provide instruction without in-context examples, ASH performs more robust then REACT. Furthermore, when combining the summarizer and actor to one prompt, the result shows that ASH’s two-step prompting method still performs better.\n\n| Model                                                             | **ASH**  Avg Score | **ASH** Success %     | **ReAct**  Avg Score | **ReAct** Success % |\n| ----------------------------------------------------------------- | --------- | --------- | --------- | --------- |\n| Codex (code-davinci-002) on 50 samples                            | 60.9      | 23.1      | 50.2 | 15.3 |\n| GPT3 (text-davinci-002) on 50 samples                             | 62.6      | 28.8      | 61.3 | 23.1 |\n| ChatGPT (gpt-3.5-turbo) with 2 in-context examples on 100 samples | 59.9      | 46.0        | 17.8 | 10.0 |\n| ChatGPT (gpt-3.5-turbo) with 0 in-context example on 100 samples | 63.6 | 43.0 | 39.4 | 3.0 |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The method proposed in this paper can be used to deal with the optimization of LLMs in complex interaction scenarios. It has certain reference significance for research in this field. The structure of the article is good, and the expression is clear and easy to read.", "reasons_to_reject": "The experiments of this work are not very sufficient, and it has not been fully verified on the current representative LLMs, and there is also a lack of corresponding ablation experiments.", "questions_to_authors": "Is it possible to test the proposed method on the text-davinci-003-based model to verify the effectiveness of the proposed method?", "paper_topic_and_main_contributions": "This manuscript proposes a method called Hierarchical Hint Assistant for optimizing the performance of LLMs on web navigation tasks. The effect of the proposed method is verified by experiments on the Webshop dataset.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "fsReXPMl57", "reviewer": "Reviewer_KCDF", "summary": "", "questions": "1. Regarding the statement on line 184, the paper lacks clarity on what is meant by the \"simpler version of dataset taken from Yao et al.\" It remains unclear if this dataset has been utilized in other research papers as well. Further elaboration and a reference to the original source would be beneficial to better understand its origin and potential usage in related studies.\n\n2. In Figure 4, how many examples, featuring more than 11 steps, are the same between REACT and ASH?", "limitations": "", "rating": 2, "confidence": 3, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thorough reviews and valuable feedback! \n\n\n\n* How SUMMARIZER works\n    * In our work how SUMMARIZER identifies and removes irrelevant information is guided by the provided in-context examples. SUMMARIZER will determine how useful an information is by considering how helpful or relevant it is for an actor to achieve the goal.\n    * Here we show one example ([https://pastebin.com/tRpTFY6k](https://pastebin.com/tRpTFY6k) ), where the SUMMARIZER eliminates irrelevant information by picking the top three relevant products in the search result. This effectively makes the observation space of ACTOR smaller, which leads to better performance. \n* Error analysis for SUMMARIZER\n    * Since we implemented our SUMMARIZER by prompting LLM, common error analysis of LLM applies here. For example, SUMMARIZER fails to summarize properly when raw observation is really long ([https://pastebin.com/CdLRC1VR](https://pastebin.com/CdLRC1VR)); SUMMARIZER hallucinates ([https://pastebin.com/bWfzSuG1](https://pastebin.com/bWfzSuG1)), adding information that does not exist in the raw observation. Besides those, ASH prompting also provides scope for improving the quality of SUMMARIZER. For example, this example ([https://pastebin.com/fz5kGHEk](https://pastebin.com/fz5kGHEk)) shows that SUMMARIZER can hint ACTOR on how to recover when it performs an Invalid Action or what it needs to do after a Think action. Such provisions can allow efficient recovery of agents compared to methods which rely solely on ACTOR prompting.\n* Difference between REACT and ASH\n    * The difference between REACT and ASH is that ASH makes two passes over the observation, removing irrelevant information in the first pass and making the final decision in the second pass. Notice that this process can be generalized to multi-pass, where we have multiple SUMMARIZERs and each of them has its own job of attending to specific information or usecases.\n    * This difference leads to a different way of processing raw observation from the WebShop environment, which is briefly mentioned in the footnote of page 3. Usually the search listing page will have 10 top search results. How REACT processes it is to just pick the top 3 and discard the remaining result, while what ASH does is to pass all 10 search results to SUMMARIZER, and the SUMMARIZER will pick the 3 most relevant search results. Due to limitations of the search engine, the top 3 of the search results might not always be the most relevant, as shown in the example provided above. In addition, longer observations have a higher chance of driving agents towards undesirable trajectories. Derivative works of REACT also follows the same setup.\n    * As for the experimental setup, we retain the complexity of the webpages differing from the ReAct paper, without any manual filtering.\n    * The main advantage of ASH is the SUMMARIZER being orthogonal to the development of better ACTOR prompts, which the existing works have majorly focused on. Infact, our main method uses ReAct based ACTOR prompting conjugated with the SUMMARIZER. We have also demonstrated using a Act based ACTOR prompting and added SUMMARIZER in the loop. In future, we expect that more sophisticated ACTOR and SUMMARIZER prompts can be individually developed and combined to achieve superlative performances. We put out our novel idea as a short paper due to this focused contribution.\n* Evaluation details and clarification for dataset setup\n    * We apologize for the missing experiments and evaluations details.The prompts we used for SUMMARIZER and ACTOR in Appendix D. We provide 4 in-context examples for the SUMMARIZER and provide 2 in-context examples for the ACTOR. Note that in Appendix C we also provided an example trajectory for ASH and an example trajectory for REACT.\n    * For the evaluation environment, we generate 500 session strings controlled by fixed seeds, as denoted below. Each session string corresponds to a specific instruction i.e. a specific task.\n\n             env = WebAgentTextEnv( # From WebShop repo\n                observation_mode='text_rich',\n                render=False,\n                num_products=None, # full product space\n             )\n\n             for seed in range(501, 1000):\n                 random.seed(seed)\n                 session = \"\".join(random.choices(string.ascii_lowercase, k=10))\n                 (ob, _) = env.reset(session=session)\n                 # test LLM agent on this session\n\n* For score calculation, there are two possibilities\n1. The trajectory ends normally i.e. it ends with the agent choosing to buy some product. In this case the environment will calculate a score between zero and one, representing the percentage of the required attributes that are satisfied by the product. When this score is 1.0, we view this trajectory as a “success” trajectory. When this score is 0.0, we view this trajectory as a “failure” trajectory”\n2. The trajectory ends abnormally. A trajectory will be terminated if (1) it produces 5 consecutive invalid actions (2) the trajectory length exceeds 20. In this case we will assign a score of 0.0 to this trajectory.\n\nWe will succinctly add the above clarifications and information to the revised paper, as necessary, for better clarity and reproducibility.\n\n\n\n* “In Figure 4, how many examples, featuring more than 11 steps, are the same between REACT and ASH?”\n    * Two methods have an (almost) non-overlapping set of examples that features more than 11 steps. Note that ASH has way more examples that feature more than 11 steps compared to REACT.\n\nThe work is aimed at complex observation space sequential tasks, like web navigation, as the title suggests and at the time of writing the paper, this was the only relevant dataset to the best of our knowledge. However, we acknowledge that the work demonstrates the method’s efficacy only on two older LLMs (including the text-davinci-002 GPT3 in Appendix B). To this front, we test our method’s generalizability to newer, stronger LLMs like ChatGPT (gpt-3.5-turbo). We choose this LLM to balance the strength and cost of running the experiment. We also chose the 4K token limit variant to test generalization to smaller token limit LLMs. While this initial analysis was run on a smaller product space with 1000 products and using 100 test samples, we will adopt the same sophisticated settings as the main results in the paper, and add it to the revised version of the paper along with the respective error analysis.\n\nAs shown in the table below, ASH surpasses ReAct’s performance on ChatGPT, mirroring the same trend as when we were using the relatively weaker Codex models or GPT3. While the major merit of ASH in this setting has been its ability to condense observation allowing longer trajectories on the smaller token limit models, its ability to help agent recover from errors and align to the optimal behavior still holds as we observed on few sample test trajectories. The pronounced improvement of success rate with ASH can also be attributed to the nature of ChatGPT agent: the agent attempts to buy the exact product and does not frequently settle with suboptimal products, compared to the older LLMs. We will also release the code and prompts for the chatgpt experiments in the same github repository as our main code.\n\n\n| Model\t| ASH Avg Score |\tASH Success % |\tReAct Avg Score |\tReAct Success % |\n| ----------| ------------------ | ------------------ | ------------------ | ------------------ |\n| Codex (code-davinci-002) on 50 samples |\t60.9\t| 23.1 |\t50.2\t| 15.3 |\n| GPT3 (text-davinci-002) on 50 samples |\t62.6\t| 28.8 |\t61.3\t| 23.1 |\n| ChatGPT (gpt-3.5-turbo) on 100 samples |\t59.9\t| 46.0 |\t17.8\t| 10.0 |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The presented SUMMARIZER prompt, which effectively filters out irrelevant information from observations, demonstrates encouraging outcomes in the context of web navigation.", "reasons_to_reject": "1. The paper lacks specific details on how SUMMARIZER effectively eliminates irrelevant information. Particularly, it does not clearly define what qualifies as irrelevant information that should be avoided when providing input to the LLM. Additionally, there is a lack of an error study on SUMMARIZER, which could help determine whether there is a possibility of crucial information being mistakenly removed by the prompt.\n\n2. The paper's failure to provide comprehensive details about the baseline models for comparison, such as REACT, is a notable limitation. The differences between ASH and REACT are not highlighted, making it challenging to discern the most crucial components of ASH that contribute to its superiority over existing approaches.\n\n3. The evaluation in the paper is not sufficiently comprehensive and lacks transparency regarding the experiment setup. For instance, there is no mention of the number of different sets of in-content examples used in the experiments. Additionally, the paper does not explore the effects of varying the number of In-Context Examples. Moreover, the evaluation relies solely on one dataset, which may limit the generalizability of the results.", "questions_to_authors": "1. Regarding the statement on line 184, the paper lacks clarity on what is meant by the \"simpler version of dataset taken from Yao et al.\" It remains unclear if this dataset has been utilized in other research papers as well. Further elaboration and a reference to the original source would be beneficial to better understand its origin and potential usage in related studies.\n\n2. In Figure 4, how many examples, featuring more than 11 steps, are the same between REACT and ASH?", "paper_topic_and_main_contributions": "The paper introduces a novel approach called Actor-Summarizer-Hierarchical (ASH) prompt for Web Navigation. Unlike previous prompting methods, the paper argues that filtering out irrelevant information from web observations before inputting them into Language Model (LLMs) is essential. The proposed ASH prompt consists of two components: SUMMARIZER and ACTOR prompts.\n\nThe SUMMARIZER prompt is designed to eliminate irrelevant details from the web observations, and the processed data is then fed into LLMs with the ACTOR prompt, which focuses on predicting the next step. ASH achieved a task success rate that outperforms the previous state-of-the-art prompting mechanism using the same LLM by 6.2%. These results demonstrate the potential of our approach in interactive decision-making tasks involving long observation traces.", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This manuscript proposes a method called Hierarchical Hint Assistant for optimizing the performance of LLMs on web navigation tasks. The effect of the proposed method is verified by experiments on the Webshop dataset.

Initially, reviewers were concerned about the representativeness of the LLMs tested. The authors addressed this issue by providing results on ChatGPT. Ablation results were also added in the rebuttal. The paper can be further improved by adding details of how SUMMARIZER works, details about the baseline models, and details of the experimental setup.
Decision: Accept-Findings