BrowserAgent: Building Web Agents with Human-Inspired Web Browsing Actions
Authors: Tao Yu
Zhengbo Zhang
Zhiheng Lyu
Junhao Gong
Hongzhu Yi
Xinming Wang
Yuxuan Zhou
Jiabing Yang
Ping Nie
Yan Huang
Wenhu Chen
Venue: Accepted by TMLR ()

Abstract: Efficiently solving real-world problems with LLMs increasingly hinges on their ability to interact with dynamic web environments and autonomously acquire external information. While recent research like Search-R1 and WebDancer demonstrates strong performance in solving web tasks, they heavily rely on additional tools to convert the interactive web environment into static text content. This is in contrast to human browsing behaviors, which involve diverse interactions with the browser, such as scrolling, clicking, and typing. In this paper, we propose BrowserAgent, a more interactive agent that solves complex tasks through human-inspired browser actions. BrowserAgent operates directly on raw web pages via Playwright through a set of predefined browser actions. We adopt a two-stage training (Supervised Fine-Tuning (SFT) and Rejection Fine-Tuning (RFT)) to improve the model's generalization abilities. Despite using significantly less training data than Search-R1, BrowserAgent achieves more competitive results across different Open-QA tasks.  Additionally, we introduce an explicit memory mechanism to store key conclusions across steps, further enhancing the model's reasoning capabilities for long-horizon tasks. Notably, BrowserAgent-7B can achieve around 20\% improvement over Search-R1 on multi-hop QA tasks like HotpotQA, 2Wiki, and Bamboogle. These results indicate that BrowserAgent can serve as a more advanced framework for more interactive and scalable web agents.

Published in Transactions on Machine Learning Research (01/2026)
BrowserAgent: Building Web Agents with Human-Inspired Web Browsing Actions
Tao Yu +1 Institute of Automation University of Chinese Academy of Sciences
y0l50 ac.cc.cr
Zhengbo Zhang
Institute of Aulomation
University of Chinese Academy of Sciences
Zhiheng Lyu +1 University of Waterloo
26llya@Qwwatertoo.coo
Junhao Gong Peking University
Hongzhu Yi University of Chinese Academy of Scienoes
Xinming Wang
University of Chinese Academy of Sciences
Yuxuan Zhou
Tsinghua University
Jiabing Yang
Institute of Automation University of Chinese Academy of Sciences
Ping Nic
ping. nie @wwatterloo
University of Waterloo
Yan Huang 1
Institute of Automation University of Chinese Academy of Sciences
Wenhu Chen University of Waterloo
wenkuchen&uuaaterloo.cu
Reviewed on OpenReview: ht sp en ew net or um Pi X4
Abstract
Efficiently solving real-world problems with LLMs increasingly hinges on their ability to interact with dynamic web enviromments and autonomously acquire external information. While recent rescarch like Search-R1 and WebDancer demonstrates strong performance in solving web lasks, they heavily rely additional tools convert the interactive web environment into static text content. This is in contrast to human browsing behaviors,
* "Equal contribution.
Work done during internship the University of Waterloo.
*Corresponding Author.

---

Published in Transactions on Machine Learning Research (01/2026)
which involve diverse interactions with the browser, such as scrolling, clicking, and typ- ing. In this paper, we propose BrowserAgent, more interactive agent that solves com- plex tasks through human-inspired browser actions. BrowserAgent operates directly on raw web pages via Playwright through a set of predefined browser actions. We adopt a Iwo-stage training (Supervised Fine-Tuning (SFT) and Rejection Fine-Tuning (RFT)) to improve the model's generalization abilities. Despite using signíficantly less training data than Search-R1, AgenttrAgent achieves more competitive results across different Open-QA tasks. Additionally, across steps, further enhancing the model's reasoning capabilities for long- horizon tasks. No- tably, Browser Agent-7B can achieve around 20% improvement over Search-RI on multi-hop QA lasks like Ao 2Wiki, and Bamboogle. These results indicate that Browser can serve as a more advanced framework for more interactive and scalable web agents.
introduce explicit memory mechanism to store key conclusions
Search-RL-instrucc
NO
Bearch-RL-instnn
NO
BrowserAgent-SFT
General QA
BrewserAgent: (LLM)
General QA
BrowserAgent-RFT (EM) Dwenn25577-hnsiruciico
Browserkgentt-RFT (LLM) Owen225577-irsstout
B
Bamboogle
PpD
0.2 0.0.0007
2 0 4.0007
Musique
HotpotIA
Musique
HolpotQA
Multi-Hop QA
Multi-Hop QA
2wiki
2wiki
Figure 1: Evaluation result of the Browseragent.
1 Introduction
Web agents [Licberman et al. 1995 Yao et al. 2022 autonomous systems that interact with vast amounts of web information to make decisions and take actions to accomplish complex real-world tasks. It has been a long-standing research field to study how to automate the complex information seekiing process. Recently, Deep Research Agents (OpenAT, 2025) from ChatGPT, Grok, and Gemini demonstrated that current LLMs exhibit unprecedented abilities to solve highly complex real-world tasks (Hendrycks et al. 2025 Mialon et al., 2023). Following these commercialized deep research systems, there has been a surging (025a)
interest in the open-source community replicate their promising results. For example, Chen et al.
Li ct al. (2025c) (Song et al. (2025): Jin et al. (2025a); Sun et al. (2025): Zheng et al. (2025 Liu et al. 2025a
have attempled various forms of SFT and RI. approaches to post-train LLMs into web agents. However, these frameworks rely heavily on external tools like HTML-parser (e.g., Jina service) and summarizers (e.g.,
prompting GPT-4o to summarize the web page) to turn dynamic web information into static short-length documents. This will (1) restrict the agents' abilities to interact with web pages to acquire in-depth information, and (2) incur very high cost due to calling the additional tools.
In contrast, humans exhibit. highly interactive browsing behaviors when solving complex web tasks. For example, Humans perform actions like hyperlink clicking, form typing, and scrolling up/down through the browser to navigate through the web for information gathering. This allows humans to acquire more in-depth and native knowledge from the web without relying on additional summarization services. Inspired by this, we resort to browser automation frameworks like Playwright to enable these interactions with dynamic DOM elements, allowing LLM agents to share the same representation space and action set as humans do.
While browser-based agents enable rich interaction, prior works such as AssistantBench (Yoran et al. 2024 WebArena ZZhou et al.] 2024), and OpenHands (Wang et. al. 2025a) primarily used browser agents for
2

---

Published in Transactions on Machine Learning Research (01/2026)
Summory
Eotrral Fsternll Sommmriiing LIM
No Web Intenntioo
Other
Came-Cininee Information
No External LLM
Click
Type
Goto
Nativ Webs Interaction
Browser Agent
Scroll
-..
Fine Grained Information
Obsevation
Action
Action List
Figure 2: Comparison between Browser A Agent and traditional deep research pipeline.
evaluation purposes. It. because Playwright camnot be efficiently parallelized. This results in extremely low throughput ( (1-2 episodes/ cerssods//mimnn minute), making large-scale training infeasible. To overcome this, we develop Ray-parallelized orchestration layer that scales to dozens of Playwright instances. On single 32-core server, our system achieves 50+ episodes/ coisodds//iintg minute, reducing the cost of browser-native data collection by over an order magnitude (see Section 2.3 for more details). Building on this infrastructure, we explore strategies for constructing high-quality, behaviorally rich training datasets directly from live browser sessions. In contrast to methods that passively snapshot every intermediate state, our system selectively maintains a structured historical memory of key conclusions while prioritizing the current observation context balancing long-term reasoning with real-time perception. This improves both sample efficiency and downstream agent performance. A vivid comparison is depicted in Figure 2
scalable, end-to-end browser- native agent framework that learns directly from real-time web interactions rather than relying on static content abstraction. BrowserAgent offers several advantages over prior work: It defines a minimal yet expressive set of atomic browser operations such as scrolling, clicking, typing, and tab management-thhr closely align with real human behavior on the web. Rather than relying on LLM-based semantic abstractions, ArenterAgent interacts directly with raw webpage states, enabling fine-grained and compositional decision-making. BrowserAgent also departs from complex reinforcement learning strategies, instead using lightweight two-stage training pipeline of SFT followed by rejection sampling. This simple yet effective approach achieves strong performanc--ssecinne hop reasoning tasks while requiring less data and infrastructure. Furthermore, its iterative ReAct-style reasoning framework, enhanced with explicit memory, empowers the agent to perform multi-turn interactions and complex inference across information sources. With only 5.3K training samples, Browser can already demonstrate strong generalization and robustness. Our model can outperform Search-R1 (Jin al. 2025b) significantly on wide range of Open-QA tasks.
We build ProwserAgent
in multi-
Through those experimental results, we demonstrate that BrowscrAgent can serve as good alternative approach to existing frameworks. Our work sheds light on how to build more scalable and interactive web agents for solving complex web tasks.
2 Browseragent
In this section, we present the detailed design of the BrowscrAgent training methodology. This includes: (1) defining a minimal yet expressive set of atomic browser operations to generate high-quality training data through interaction with dynamic web environments, ensuring that the agent learns how engage with real-world web content; and (2) training BrowserAgent using efficient SFT and RFT enable it to handle complex reasoning tasks. Our approach focuses on leveraging real-time web interaction instead of relying static datasets, providing more flexible and scalable train-
3

---

Published in Transactions on Machine Learning Research (01/2026)
ing framework. By systematically generating data from interactive search behaviors, have successfully enhanced the model's problen-solving capabilities and its ability to handle diverse, real-world information.
Train
City
SFT Data
Rollout
Tool Server
Browser
Original RFT Data
RFT Data
Base Model
SFT Model
RFT Model
Inference
In which UK city the long-rum British scao opera in which Abby Mavers' farther appears as Gordon Livesy filmed?
Observation,
Thinking,
Memory,
Action,
textiace "Search kikipedia"
anaver the cuestion, need filsing leng-runeing teitish seep opers features sondon Liveyy.
ttp://
'welcome
Liveyy Ewrerdale" Betets hi soop opera.
[198]
tuttonn &
type erdele]
2122 search box sok up desatis about the show, porticaliely filnting
[355
bettoon vokipedie"
Browser,
Q
stepe
Thinking,
The accessibility reveals rilaed at "the Leeds Studies" since its incuption, and bu sarewsdd extatis. reeds is csty Therefore, the
Browser,
the search results for "tn mediie" is open, and the first meedsie article, articce is iteely to coneain seformatio Hicking Lirk ticle steuld provide the necess ary detadin.
amserr qu eation, need location.
elick [12M]] [f
mmedaaee]
stop [Lawds]
Memonys
Action,
Thinking,
Answer
Figure 3: Overview of the Browseragent framework.
2.1
Framework Design
First, we simulate real human operations by defining four categories of actions: page operation actions, tab management actions, URL navigation actions, and completion actions. The Browser Agent action categories and their corresponding command descriptions are provided in Table These categories are designed comprehensively cover the full spectrum human-web interaction behaviors. Unlike other approaches, we do not rely on any additional models for web parsing and summarization (Wu al., 2025). Instead, simulate human browsing behavior through direct interactions with the web page- such scroll up/down actions in the page operation category enteliry-eaabling the model to gradually develop the ability to autonomously locate required information and understand web content during training. This design brings multiple advantages. Fine-grained page operation and (ab management actions not only enhance the model's understanding of web structures and its adaptability to multi-task scenarios, but also significantly improve the agent's generalization ability in complex web environments. Furthermore, by simulating human exploratory behaviors through actions like scroll up/down, the model is compelled actively filter, locate, and integrate key information during real interactions. This avoids shortcuts such relying on additional

---

Published in Transactions on Machine Learning Research (01/2026)
Table 1: BrowserAgent action categories and command descriptions
Human Operation Cat- egory
Action Commands
Descriptions
plick(id, content) hover hover(id, content)
Executes a click on the element specified by the Hover the cursor over the element associated with the id.
Page Operation Actions
press (key_comb)
Simulates the pressing of a key combination on the keyboard (e.g,, Ctrl+v) Scroll the page up or down. Inputs content into the designated field (optionally followed by an Enter keystroke).
scroll (down lup) type(id, content, press_ enter ttr___eteerr_rrreee after=01
new new_tab tab bab_focus (tab_itdee)) index)
Open a new, empty browser tab. Switch the browser's locus to a tab using its index. Close the currently active tab.
Tab Management Actions
close_tab tab
goto
Navigate to a specific URL.
URL Navigation Actions
go back
Navigate to the previously viewed page.
go go_forward
Navigate to the next page (if a previous _go__aaall back'
action was performed).
Completion Action
stop (answer)
Concludes the interaction by providing an explicit answer or designating the outcome as "N/A."
large model for summarization, and gradually builds the model's capabilities in information retrieval, reading comprchension, and content integration throughout the training process.
In addition, ing summarizing acting. Conceptually, this paradigm be regarded as variant of the ReAct frame- work (ao et al. 2023bb augmented with a memory module, enabling the agent to maintain contextual continuity across multiple interaction turns. Specifically, both data generation and model inference are con- ducted through our automated multi-turn interaction process. We provide the system prompt and question, interact with the server to obtain observations, and then provide the prompt, question, and observations GPT-4.1 (OpenAI 2024). which outputs the (reasoning, action) pairs like ReAct (Yao et al. 2023b). Additionally, we require the model to record necessary conclusions. This process is iterated until the model outputs answer to the question or the number of steps exceeds the limit. We keep track of historical actions and conclusions as memory in the input to help the model understand the completed actions and gather necessary information. The complete workflow is outlined in Algorithm [ The prompt is provided in the Appendix A.3
simulate human search behavior through an iterative cycle of think-
2.2 Dataset Synthesis
To equip the model with comprehensive problem-solving capabilities, we selected the general question answer- ing dataset NQ (wwitkkoskii et al. 2019 and the multi-hop question answering dataset HotpotQA (Yang et al. 2018). These two datasets cover basic factual questions and tasks that require complex reasoning and multi-step inference, helping the model learn richer reasoning paths and answering strategies different types of problem sccnarios. Our datasct consists of a total of 5.3K high-quality trajetories. Among them, 4K routes come from NQ and HotpoiQA for simple question-answering tasks. The rest 1.3K trajetories are selected from HotpotQA to specifically improve the model's ability to tackle complex multi-hop reasoning tasks. All selected samples are processed through our automated multi-turn interaction collection process. The detailed procedure is described in Section 2.1
For basic capability data, the system drives the large model to interact with the web environment for up to steps, and for challenging data, it drives the large model interact with the web environment for up 30 steps. Every step fully recorded, including the input prompt, model output, current page observation, and intermediate conclusions. This process ultimately generates detailed reasoning and decision
5

---

Published in Transactions on Machine Learning Research (01/2026)
trajectory data. Our case is provided in the Appendix A.4 Through this dual selection and collection strategy, our dataset not only ensures broad coverage of the model's basic capabilities but also specifically enhances its performance in complex, multi-hop reasoning scenarios, laying a solid foundation for subsequent agent training and evaluation.
2.3 Environment Setting
Based on Verl-Tool (Jiang al. 2025 and WebArena Zhou et al. 2024): we implement playwrighh-based tool to mimic the interaction with the browser. The observation is pure text-based, an accessibility tree parsed by Playwright (Microsoft] 2020) (scc example in Figure 3). To improve page rcadability, we do some rule-based post-processing such as merging consecutive text elements on the accessibility tree. To deploy Wikipedia (Karpukhin et al., 2020) locally, we adopt the offline Kiwix version with content updated to 2022. To standardize the interface and accelerate interactions, we employed the Verl-Tool framework with Ray- Ray-based (Moritz et al. 2018) tool server as the top-level interface. All requests were routed through FastAPI (Ramírez) 2018), enabling uniform scheduling by the backend. Based the design of Verl-Tool's tool server, each episode is assigned unique trace id to maintain the correspondence with its browser session, and the session is aulomatically terminated once a stop action is received. In our experiments, we deployed 64 concurent Playwright browsers on single machine with 32-CPUs. While the top-level scheduling is orchestrated by Ray (Moritz et al.) 2018 the framework exhibits strong potential for scaling to much larger CPU clusters.
2.4 Training
To enhance the model's ability to generate high-quality, structured answers, propose a two-stage training framework comprising SFT and RFT, as illustrated in the Figure 3
Stage 1: SFT. In the first stage, we perform SFT on the base model Qwen2.5-7B-IInstuucl using 5.3K question-answer pairs generated via the method described in Section 2.2 This stage is designed to teach the model the desired answer format and to equip it with basic reasoning capabilitics. The model is trained until convergence, and the checkpoint with the best evaluation loss selected as the final SFT model. The evaluation loss curve is shown in the Appendix A.I The primary goal of the SFT stage is to establish an initial understanding of the answer structure.
Stage 2: RFT with Reasoning Path Selection. In the second stage, we leverage the SFT model obtained from the first stage to construct training data for reinforced fine-tuning. Specifically, for each question in the training sets of NQ and HotpotQA, we sample four answers from the SFT model. We then use the EM metric for filtering. following the approach in (Xiong al., 2025), to select instances where the model-generated answers include both correct and incorrect responses among the four candidates. From these instances, choose the correct answer that contains the largest number of reasoning steps (Zhang et al., 2025), as more reasoning steps indicate deeper model thinking. This encourages the model to learn richer and more in-depth reasoning patterns. The selected samples constitute our Original RFT Data. To preserve the distributional characteristics of the original data and mitigate the issue of catastrophic forgetting of the answer format learned during the SFT stage, we randomly sample a portion the SFT data and combine it with the filtered Original RFT Data to construct the final dataset used in the second-stage training. We then further fine-tune the SFT model on this mixed dataset, thereby enhancing its reasoning capability while maintaining its ability to generate well-structured answers. The specific data ratio and filtering strategy are detailed in the Appendix A.5
3 Experiments
3.1 Benchmarks
To comprehensively evaluate the performance of Browscragent, we selected six representative benchmark datasets, which are divided into two categories: The first category is general question answering datasets, including NQ (Kwiatkowkki et al. 2019), and PopQA (Mallen al. 2023); the second category is multi-hop
6

---

Published in Transactions on Machine Learning Research (01/2026)
Algorithm Rollout/Infereee Pipeline
Require: Input question q, BrowserAgent model ne, VerlTool tool server V, webpage w, obsorvation from webpage o, history actions A, memory M, maximumn action sleps S, trace id uid-
Ensure: Final answer ans. Wikipediia V(w, 0)
1: Initialize webpage
Initialize observation
3: Initialize history act ions A < 0
4: Initialize memory M < 0
5: Initialize action count & +
6: Initialize trace id uia + rand()
while S do Generate response y ~ (4 Os. M) if <conclusion> </conclusion> detected in y then Extract m, = Parsely, <conclusion>, </conclusion>) Insert m, into memory M + M m, end if
10:
11:
12:
13:
" "comand [parameters] "C detected in then
14:
Extract a, = Parsely, "< "() Insert a into history actions A A 0s if stop equal to comand then
15:
16:
17:
return Final answer parameters end if
18:
19:
else
20:
Set ag =
21:
end if
22:
Reset observation from webpage os+1 V(u,, ag, usa) Increment action count +- ++1
23:
24:
end while
question answering datasets, including HotpotQA (Yang al. 2018 2WikiMultHHoQAA (oo al. 2020
Musique (Trivedi et al., 2022), and Bamboogle (Press et al. 2023 These datasets encompass a diverse range of search and reasoning tasks, enabling systematic and comprehensive evaluation of Browseragent.
3.2 Evaluation Metrics
In our experiments, we adopted two evaluation methods: exact match (EM) Yu et al. 2024] and LLM- based judgment. FM offers an objective measure consistency between model outputs and ground truth, but often underestimates performance when answers are factually correct and semantically reasonable yet differ in surface form (see case study in Appendix A.4). To complement this, introduced an ALM-based judgment mechanism: each model-generated answer was independently evaluated by GPT-4.1 (OpenAT 2024h, Gemini Flash 2.5 [DeepMind] 2023), and Claude Sonnet 3.7 (Anthropic, 2024). and classified as "valid" " if at least two judged it correct. By combining EM with consensus voting among multiple LLMs, our framework provides a more comprehensive and reliable view of model performance, mitigating the limitations of single-metric evaluation in complex, open-domain QA tasks. The full evaluation prompt provided in
Appendix A.2
3.3 Baselines
To evaluate the effectiveness BrowserAgent, we compared it with the following baseline methods, adopting the same settings as those used in Scarch-R1 (Jin et al.] 2025bb:
(1) Inference without Retrieval: Direct
inference and Chain-of-Thome Augmented Generation (RAG) [Lewis et al. 2021 IRCoT Trivedi et al. 2023), and Search-o1 (Li et al.
(CoT) reasoning (Wei al. 2023). (2) Inference with Retrieval: Retrieval-

---

Published in Transactions on Machine Learning Research (01/2026)
2025d (3) Fine Tuning- Based Methods: SFT CChung et al. 2022 (4) RL-based methods: Search-R1 et al. 2025b).
Jin
These baselines cover broad range of paradigms in retrieval augmentation and fine-tuning, providing a representative comparison for evaluating BrowserAgent. For fairesss, we use the same pre-trained LLM across all methods and align the training data sources with those of the baselines. Although our training scale is much smaller than those used in prior work, BrowserAgent still achieves non-trivial performance gains.
3.4 Experimental Setup
We conduct our experiments using the Qwenn2.577-Instruuc model (Team 2024 The 2022 Wikipedia dump (Karpukhin al. 2020) is used the knowledge source. During evaluation, we limit the model to maximum of 30 rounds of browser interactions. Other training details can be found in the Appendix A.6
3.5 Experimental Results
Based the experimental results in Table find that Arewser consistently outperforms strong baselines across range of tasks, demonstrating robust performance in both in-distribution and out-of- distribution evaluations. Our RFT stage achieves consistent improvements over the SFT stage, particularly on multi-hop QA tasks. Notably, Browser significantly outperforms multiple versions of Search-R1. Specifically, Agent-777 can achieve around a 20% improvement over Search-R1-Instrun on multi-hop QA tasks. This advantage stems from its ability to handle longer reasoning chains without being constrained by context length. Even with limited training data and only a simple, efficient RFT strategy, still out- performs Search-R1-Insrruce trained with complex reinforcement learning techniques, fully demonstrating the effectiveness and practicality of BrowserAgent's end-to-end paradigm based on direct interaction with web pages in lasks involving aclive web exploration and complex reasoming. Moreover, as RFT does not use explicit format-based rewards, EM alone may not fully capture performance; incorporating LLM-based judgment provides a more comprchensive evaluation of the model's true capabilitics.
Main results baselines and our methods. The pest-perfooming results for EM and LLM-based judgment are highlighted bold.
Table 2:
General QA
Multi-Hop QA
Methods
Avg.
NQ
PopQA
HotpotQA
2wiki
Musique
Bamboogle
IND/OOD
IND
OOD
IND
OOD
OOD
OOD
Search-R1-DD R1-Direct Inference
0.134
0.140
0.183
0.250
0.031
0.120
0.143
Scarch-ol
0.151
0.131
0.187
0.176
0.058
0.296
0.167
Search-R1-CoT
0.048
0.054
0.092
0.111
0.022
0.232
0.093
Search-RI-IRCoT
0.224
0.301
0.133
0.149
0.072
0.224
0.184
Search-RI-RAG
0.349
0.392
0.299
0.235
0.058
0.208
0.257
Search-R1-SFT
0.318
0.121
0.217
0.259
0.066
0.112
0.182
0.393
0.397
0.370
0.414
0.146
0.368
0.348
Ours
Qwea2.5-7b-Instrucc (EM)
0.146
0.244
0.138
0.097
0.039
0.080
0.124
Browseragent-SFT (EM)
0.371
0.437
0.441
0.500
0.157
0.456
0.394
BrowseragenttRRFT (EM)
0.388
0.431
0.458
0.498
0.164
0.504
0.407
-wea2..--b-Inntruect (LLM-judge)
0.191
0.279
0.186
0.154
0.054
0.096
0.160
Browseragent-SFT (LLM-judge)
0.466
0.487
0.547
0.599
0.204
0.504
0.468
Browscragent-RFFT (LLM-judge)
0.493
0.485
0.561
0.601
0.212
0.552
0.484
8

---

Published in Transactions on Machine Learning Research (01/2026)
3.6 Ablation
For the ablation studies, randomly sampled 1K examples from each benchmark dataset for evaluation. As Bamboogle contains fewer than 1,000 instances, we evaluated it in full. Since RFT models trained on poorly performing SFT models tend to exhibit even lower performance, we conducted most ablation experiments on the SFT model, while using the RFT model specifically for step-wise ablations. In this section, we report only the FM scores as the evaluation metric.
Increasing Memory Capacity. For instance, when answering a question like "Who the lather of the greatest NBA player of all time?", the model may first determine that "Michael Jordan is the greatest NBA player" and then retrieve that "Jordan's father is James Jordan." Without storing the intermediate conclusion, the model risks losing the connection to the original question and falling into circular search pattern. To enhance reasoning efficiency and avoid redundant steps, we increased the model's memory capacity by training it to summarize and retain intermediate conclusions during multi-hop reasoning. The experimental results in Table 3 show that the proposed method not only effectively prevents information loss and logical disconnection reasoning chains, but also improves the model's overall performance and gencralization when handling complex tasks.
Table 3: Results of the ablation study on BrowscrAgent-SFT.
General QA
Multi-Hop QA
Memory
Single-round
Steps
Parameters
NQ
PopQA
HotpotQA
2wiki
Musique
Bamboogle
Avg.
IND
OOD
IND
OOD
OOD
OOD
6
7B
0.316
0.420
0.382
0.385
0.130
0.368
0.334
6
3B
0.298
0.362
0.335
0.359
0.087
0.264
0.284
6
7B
0.332
0.420
0.384
0.423
0.131
0.360
0.342
x V
30
0.341
0.417
0.399
0.415
0.132
0.384
0.348
30
7B
0.387
0.466
0.434
0.457
0.152
0.456
0.392
Multi-round observation v.s. Single-round observation Previous argentic posttraining works such as RAGEN (Wang et al., 2025b) and Search-R11 treat the entire tool-use trajectory as a single input, appending all retrieved content and intermediate reasoning steps into the context. This approach is limited by the context window, restricting scalability during inference. To address this, we introduce memory mechanism that stores only key conclusions in a structured <memory>> format, removing redundant context. while preserving essential information. This enables the model to reason more efficiently and scale to up to 30 reasoning steps without exceeding context limits. The experimental results in Table 3 demonstrate that trimming redundant input not only improves model performance, but also enhances its scalability in handling complex multi-hop tasks.
Model Size Scaling Due to computational resource constraints, we conducted comparative experiments using models with 3B and 7B parameters. As shown in the Table 3 the 7B model consistently outperforms the 3B model across all evaluation benchmarks, demonstrating stronger reasoning capabilitics and better generalization. This result provides compelling evidence for the importance of scaling model size in multi-hop reasoning and complex question answering tasks. Furthermore, it highlights that even under limited training data and relatively lightweight fine-tuning procedures, model capacity remains critical factor influencing overall performance.
Test Step Scaling Leveraging our single-turn context window architecture, the proposed method scales effectively by increasing the number of interaction rounds. While performance is slightly lower than Search- R.1 at around turns, it improves steadily as the mumber of turns grows. Notably, even with a significantly increased maximum step limit (c.g., steps), the average number of reasoning steps remains stable, in- dicating the model's ability to complete tasks efficiently without overusing the step budget. The detailed experimental results be found in Tables and 5 in the Appendix A.7.1
9

---

Published in Transactions on Machine Learning Research (01/2026)
4 Related Work
4.1 Rejection sampling Fine-Tuning
Rejection Sampling Fine-Tuuing (Dong et al. 2023 is a widely adopted strategy for enhancing large language models (LLMs), particularly when integrating generative models with external evaluation criteria or retrieval mechanisms. In this approach, the model generates multiple candidate outputs (Yuan et al. 2023 Xiong ct al.
2025 Huang et al. 2022 for each input. These candidates are then assessed using predefined evaluation metrics such as answer correct or adherence to specific constraints (Yang et al. 2024]. Only the top- performing responses, determined by the evaluation criteria, are retained through rejection sampling. The rosulting high-quality samples are subsequently used to further finc-tune the model (Zelikman al. 2022)
This method has been shown to effectively improve the accuracy and robustness of LLMs across various tasks, including question answering, code generation (Li et al. 2025ap, and reasoning (Team 2025). We extend this method to the domain of browser agents and achieve strong performance on several key tasks, demonstrating its broad applicability and effectiveness in dynamic web environments.
4.2 Agentic Information Seeking
Agentic Information Seeking is retrieval approach in which agent actively participates in acquiring and integrating information. Unlike traditional keyword search, this method allows the agent to behave
like a human user, automatically performing a series of operations in browser or web enviromment such as Lyping, clicking, and navigating to progressively collect and filter useful information. (de Chezelles
iet al. 2025 Nakano et al. 2022 Schick et al., 2023 Jiang et al. 2025 Murphy et al., 2025) It is typically powered by large language models or automation strategies, enabling the agent to understand complex lasks, dynamically adjust its actions, and synthesize information from multiple sources to meet. complex multi-turn information needs. Although current approaches (Jin et al. 2025b Wu et al. 2025) make notable progress in this field, they still face several limitations for instance, their limited capacity to model lontecontext information, which results in a heavy reliance on external resources such as the Jina service and larger-scale pretrained models. To address these challenges, we propose a strategy that integrates web page scrolling operations (e.g., scroll up and down) into an end-to-end model, enabling the agent to actively read web content. This approach effectively alleviates the issues above and enables efficient understanding of complex web information.
4.3 Agentic Search
Web Agents are intelligent agent systems capable of autonomously interacting with internet resources ctt al. 2025c b Liu et al. 2025b), such as web pages and search engines, to accomplish complex tasks includ- ing multi-slep reasoning, information retrieval, question answering, and data integration. Typically, these systems leverage LLMs as their decision-making core, in combination with tool invocation, web page parsing, retrioval, and automated exccution functionalities. In recent years, Web Agents have bocome critical foun- dation for research in open-domain problem solving, vetrieval-auggenntel generation (Chen et al. 2025b). AI assistants, and automated search-based reasoning. Recent works mainly focus on benchmarking (Zhou et al. 2024 Yao et al.) 2023aa Koh et al. 2024 Yoran et al., 20244] Yang et al. 2025) rather than training agents di- rectly in these environments, primarily because browser- based interaction is computationally expensive. To this end, we have implemented an efficient and easy-to-use two-stage browser agent training pipeline, which ensures scalability while significantly reducing the system overhead of environment interactions, thereby laying the foundation for large-scale aulonomous learning of Web Agents.
Li
5 Conclusion and Future Work
This paper presents BrowserAgent, an efficient and scalable training framework for real-world web interac- tion. Unlike previous approaches that rely on static summarics or external parsing services, our method leverages Playwright to enable end-to-end web operations, allowing the model actively acquire and in- tegrate information through fine-grained actions such as scrolling and clicking. Combined with parallel
10

---

Published in Transactions on Machine Learning Research (01/2026)
tool serving architecture and explicit memory mechanism, two-stage SFT + RFT training strategy reduces redundant context while enabling efficient scaling along complex reasoning chains. Experimental and ablation results demonstrate that BrowserAgent achieves significant advantages in performance, scalability, and data efficiency. Notably, BrowserAgent outperforms existing methods (c.g., Scarch-R1) using a smaller amount of Training data, especially excelling multi-hop reasoning tasks.
In future work, we plan to explore more intelligent memory mechanisms, cross cross-website website generalization, multi- agent collaboration, and contimual learning from interaction logs to advance BrowserAgent toward truly ceneral-puppor web agent.
Acknowledgment
The authors would like to thank Jinbo Su for his helpful discussions. This research was supported in part by the Google Cloud Research Credits Program.
References
Anthropic. Claude 3 models. httppp//www.attttoc.ccccc///aaaa
2024.
Accessed: 2025-09-21.
Mingyang Chen, Tianpeng Li, Haoze Sun, Yijie Zhou, Chenzheng Zhu, Fan Yang, Zenan Zhou, Weipeng Chen, Haofen Wang, Jeff Z Pan, et al. Learning to reason with search for llms via reinforcement learning. arXiv preprint ar 25x: 19470, 2025a.
Zijian Chen, Xueguang Ma, Shengyao Zhuang, Ping Nic, Kai Zou, Andrew Liu, Joshua Green, Kshama Patel, Ruoxi Meng, Mingyi Su, Sahel Sharifymoghaddam, Yanxi Li, Haoran Hong, Xinyu Shi, Xuye Liu, Nandan Thakur, Crystina Zhang, Luyu Gao, Wenhu Chen, and Jimmy Lin. Browsecomp-plas more fair and transparent evaluation benchmark of deep-roscarch agent. aixiv preprint or 2008v:2508. 066000 2025b.
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V. Lc, and Jason Wei. Scaling instruction-fieeuum
language models, 2022. URL https://arxiv.cr hanggg/bba/2a ab10/ 11416
Thibault Le Sellier de Chezelles, Maxime Gasse, Alexandre Lacoste, Massimo Caccia, Alexandre Drouin, Téo Boisvert, Megh Thakkar, Tom Marty, Rim Assouel, Sahar Omidi Shayegan, Lawrence Keunho Jang, Xing Han Lů, Ori Yoran, Dehan Kong, Frank F. Xu, Siva Reddy, Graham Neubig, Quentin Cappart, Russ Salakhutdinov, and Nicolas Chapados. The browsergym ecosystem for web agent rescarch. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URI. https::/openreviiee http:////ppparrreeeeeeee////e/ee net/forum?id=5298
FKGmv3 Expert Certification.
Google
DeepMind.
Gemini:
Google's
multimodal
models,
2023.
how.lidddgooog//eeeeeeeeeeeeeeee (technologies/geminne
Hanze Dong, Wei Xiong, Deepanshu Goyal, Yihan Zhang, Winnie Chow, Rui Pan, Shizhe Diao, Jipeng Zhang, KaShun SHUM, and Tong Zhang. Raft: Reward ranked finetuning for generative foundation model alignment. Transactions on Machine Learning Research, 2023.
Dan Hendryeks, Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Sean Shi, Michael Choi, Anish Agrawal, Arnav Chopra, Adam Khoja, Ryan Kim, Richard Ren, Jason Hausenloy, Oliver Zhang, Mantas Mazeika, Summer Yue, and Alexandr Wang. Humanity's 'ss last exam. arXiv preprint 2rXiv:2501.14249, 2025. URL https:://tssiiiii org/ atspaa/....2aaaaa 14249.
Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps, 2020. URL.
11

---

Published in Transactions on Machine Learning Research (01/2026)
Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. Large language models can self-improve 2022. URL https://arxxv..rrr atgg/bs//210.... 2/t./221.....1
Dongfu Jiang, Yi Lu, Zhuofeng Li, Zhiheng Lyu, Ping Nie, Haozhe Wang, Alex Su, Hui Chen, Kai Zou, Chao Du, et al. Verltool: Towards holistic agentic reinforcement learning with tool use. arXie preprint arXiv:2509.0100 01055, 2025.
Bowen Jin, Hansi Zeng, Zhenrui Yue, Dong Wang, Hamed Zamani, and Jiawei Han. Search-r1: Training 1lms to reason and leverage search engines with reinforcement learning. Xiv preprint or 2a:25033009116 2025a.
Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Scarch-rl: Training Ilms to reason and leverage search engines with reinforcement learning, 2025b. URL https://arxiv orgg http//bb/22500 abs/2503. 09516
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen tau Yih. Dense passage retrieval for open-domain question answering, 2020. URL https:
//axxiv.or//as//2044.... http:/abs/200..04900
Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Chong Lim, Po- Yu Huang, Graham Neubig, Shuyan Zhou, Ruslan Salakhutdinov, and Daniel Fried. Visualwebarena: Evaluating multimodal agents realistic visual web tasks. arXiv preprint 241 2024.
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming- Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. Natural ques- tions: benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:452-466, 2019. doi: 10.1162/tacl a 00276. URL pttp:ss//cclaattoollgggg wwwp:tttooooooooooooggge www.oog-////// 1026/
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen lau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval- augmented generation for snowledge-intensiv nlp tasks, 2021. URL
114
01
Haochen Li, Wanjin Feng, Xin Zhou, and Zhiqi Shen. Gift: Gibbs fine-tuning for code generation, 2025a.
URL https://arxiv. org/ 2a02./2500..00 11466
Kuan Li, Zhongwang Zhang, Huifeng Yin, Liwen Zhang, Litu Ou, Jialong Wu, Wenbiao Yin, Baixuan Li, Zhengwei Tao, Xinyu Wang, Weizhou Shen, Junkai Zhang, Dingchu Zhang, Xixi Wu, Yong Jiang, Ming Yan, Pengjun Xie, Fei Huang, and Jingren Zhou. Websailor: Navigating super-human reasoning for web agent, 2025b. URL btttts://aaiiiii wwwaaawwwv....vaaaeeeeeeaeeeeeee ht///bbb/250000 .22592.
Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia. Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. Search-ol: Agentic search-enhanced large reasoning models. arXiv preprint ar Xix:2201.005300 2025c.
Xiacxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. Search-ol: Agentic search-enhancod large reasoning modcls, 2025d. URL
3/2501.05366
Xiaoxi Li, Jiajie Jin, Guanting Dong, Hongjin Qian, Yutao Zhu, Yongkang Wu, Ji-Rong Wen, and Zhicheng Dou. Webthinker: Empowering large reasoning models with deep research capability, 2025e. URL https:
//arxiv. org/ abs/2504.... 21776
Henry Lieberman et al. Letizia: An agent that assists web browsing. IJCAI (1), 1995:924 929, 1995.
Junteng Liu, Yunji Li, Chi Zhang, Jingyang Li, Aili Chen, Ke Ji, Weiyu Cheng, Zijia Wu, Chengyu Du, Qidi Xu, Jiayuan Song, Zhengmao Zhu, Wenhu Chen, Pengyu Zhao, and Junxian He. Webexplorer: Explore and evolve for training long-horizon web agents. Xiv preprint arXiv:2509.0650, 2025a.
12

---

Published in Transactions on Machine Learning Research (01/2026)
Junteng Liu, Yunji Li, Chi Zhang, Jingyang Li, Aili Chen, Ke Ji, Weiyu Cheng, Zijia Wu, Chengyu Du, Qidi Xu, Jiayuan Song, Zhengmao Zhu, Wenhu Chen, Pengyu Zhao, and Junxian He. Webexplorer: Explore and evolve for training long-horizon web agents, 2025b. URL
Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannanch Hajishirzi. When not trust language models: Investigating effectiveness of parametric and non-parametric memories, 2023. URL
Gabriel Mialon et al. Gaia: benchmark for general ai assistants. Xiv preprint arXiv:2311 129998,, 2023.
Microsoft. Playwright. http:s//ggthhl.cco///aaoooooof/g 2020. Accessed: 2025-09-21.
Philipp Moritz, Robert Nishihara, Stephanic Wang, Alexcy Tumanov, Richard Liaw, Eric Liang, Melih Elibol, Zongheng Yang, William Paul, Michael I. Jordan, and lon Stoica. Ray: A distributed framework for emerging ai applications, 2018. URL
wttp://.iwrrgrb//////a 1711205889
Alexander Murphy, Mohd Sanad Zaki Rizvi, Aden Haussmann, Ping Nie, Guifu Liu, Aryo Pradipta Gema, and Pasquale Minervini. An analysis of decoding methods for Ilm-based agents for faithful multi-hop question answering, 2025. URL https://arxiv. org/ abs/ abaaabaa//500/ 2503. 23415
Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Floundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. Webgpt: Browser-assisted question-answering with human feedback, 2022. URL /ttpps://axiv..ttt
OpenAl. Gpt-4 technical report, 2024. URL
OpenAL. Openai o3 and o4-mini system card. Technical report, OpenAI, April 2025. URL https://cd openaii
card pdf
Version 2 of Preparedness Framework.
Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A. Smith, and Mike Lewis. Mcasuring and narrowing the compositionality gap in language models, 2023. URL
50
Sebastián Ramírez. Fastapi. https://github. http:///ww...mm//immmmo///ss/t/p 2018. Accessed: 2025-09-21.
Timo Schick, Jane Dwivedi- Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer,, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools, 2023.
URL /uttp:://arxit org/ ab3/202 2201//222220070111
Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen. R1-searcher: Incentivizing the search capability in lims via reinforcement learning, arXiv preprint Xrx::25003. 05592, 2025.
Shuang Sun, Huatong Song, Yuhao Wang, Ruiyang Ren, Jinhao Jiang, Junjie Zhang, Lei Fang, Zhongyuan Wang, Wayne Xin Zhao, and Ji-Rong Wen. Simpledeepsearcher: Deep information seeking via web- powered reasoning trajectory synthcsis. GitHub: 2025.
NovaSky Team. Sky-tl:
Train your own ol preview model within 450.
htlpp://novasky-
ai.github.io/ teabbboooooossss--ttteee 2025. Accessed: 2025-01-09.
Qwen Team. Qwen2.5: A party of foundation models, September 2024. URL https://quenlm. wwt.mmm.git.......m i0 2allog/wen2.5/
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Musique Multihop ques- tions via single-hop question composition, 2022. URL http:////rrai...rrr//b////000000
13

---

Published in Transactions on Machine Learning Research (01/2026)
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions, 2023. URT. https://arxiv..r
g/abs/2212.1050 10509
Xingyao Wang, Boxuan Li, Yufan Song, Frank F. Xu, Xiangru Tang, Mingehen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H. Tran, Fugiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Yanjun Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, Junyang Lin, Robert Brennan, Hao Peng, Heng Ji, and Graham Neubig. Openhands: An open platform for AT software developers as generalist agents. In The Thirteenth International Conference on Learning Representations, 2025a. URL https: //openreview.o weweanreveeennetffoooooee net//forum2id--OJdaa//DDo
Zihan Wang, Kangrui Wang, Qineng Wang, Pingyue Zhang, Linjie Li, Zhengyuan Yang, Xing Jin, Kefan Yu, Minh Nhat Nguyen, Licheng Liu, Eli Gottlieb, Yiping Lu, Kyunghyun Cho, Jiajun Wu, Li Fei-Fei, Lijuan Wang, Yejin Choi, and Manling Li. Ragen: Understanding evllutvolution in Ilm agents via multi-turn reinforcement learning, 2025b. URL ltopp:///arivya
20073
Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoe Le, and Donny Zhou Chain-of-thouuht prompting clicits reasoning in large language models, 2023. URL
atttps//arrixs wwwrxii..or/gbbai wttg/ba/////eeeeeeoooeeeee argg/aas/2001.1111g 2ttp:/221..11900 11903
Jialong Wu, Baixuan Li, Rumnan Fang, Wenbiao Yin, Liwen Zhang, Zhengwei Tao, Dingchu Zhang, Zekun Xi, Gang Fu, Yong Jiang, Pengjun Xie, Fei Huang, and Jingren Zhou. Webdancer: Towards autonomous information seeking agency, 2025. URL https: htt/aaaii...rg/bbaaaaaaaaaaaaaaa wrg.ooo////aaaaaeeooooe 2505. 22648
Wei Xiong, Jiarui Yao, Yuhui Xu, Bo Pang, Lei Wang, Doyen Sahoo, Junnan Li, Nan Jiang, Tong Zhang, Caiming Xiong, and Hanze Dong. A minimalist approach to Ilm reasoning: from rejection sampling to reinforce, 2025. URL https://arxiv wrg. 11343
An Yang, Beichen Zhang, Binyuan Hui, Bofci Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, and Zhenru Zhang. Qwen2.5-math technical report: Toward mathematical expert model via self-improvemen,, 2024. URL httpss://axxi...... owwpexiii..oraaaaoe htwpg///b///4/....e/// 2409 12122
Jialin Yang, Dongfu Jiang, Lipeng He, Sherman Siu, Yuxuan Zhang, Disen Liao, Zhuofeng Li, Huaye Zeng, Yiming Jia, Haozhe Wang, Benjamin Schneider, Chi Ruan, Wentao Ma, Zhiheng Lyu, Yifei Wang, Yi Lu, Quy Due Do, Ziyan Jiang, Ping Nie, and Wenhu Chen. Structeval: Benchmarking Ilms' capabilities to generate structural outputs, 2025. URL hhttps:///art:t waarxiv.org//eee htsp/bbb/2555..
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering, 2018. URL
htsp//bb//1809 1809009600
Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. Webshop: Towards scalable real-world web interaction with grounded language agents. Advances in Neural Information Processing Systems, 35: 27744, 2022.
Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. Webshop: Towards scalable woaldworld web interaction with grounded language agents, 2023a. CRL httlhhhtss/aaaat http://rxiivvorga///220///
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models, 2023b. ICLR 2023.
Ori Yoran, Samuel Joseph Amouyal, Chaitanya Malaviya, Ben Bogin, Ofir Press, and Jonathan Berant. Assistantbench: Can web agents solve realistic and time-consuming tasks?, 2024. URL https://arxiv. org/ ht//abbs/4077 2407 15711
Yue Yu, Wei Ping, Zihan Liu, Boxin Wang, Jiaxuan You, Chao Zhang, Mohammad Shocybi, and Bryan Catanzaro. Rankrag: Unifying context ranking with retrieval-angnnented generation in llms, 2024. URL
https://arxiv wwt.v/.rgg.//////eee...... (ttaabb/224770220
14

---

Published in Transactions on Machine Learning Research (01/2026)
Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Keming Lu, Chuanqi Tan, Chang Zhou, and Jingren Zhou. Scaling relationship on learning mathematical reasoning with large language models, 2023.
URL attps://axx.vvorrr/b///333. 01825
Eric Zelikman, Yuhuai Wu,, Jesse Mu, and Noah D. Goodman. Star: Bootstrapping reasoning with reasoning, 2022. URL
/2003. 14465
Dingchu Zhang, Yida Zhao, Jialong Wu, Baixuan Li, Wenbiao Yin, Liwen Zhang, Yong Jiang, Yufeng Li, Kewei Tu, Pengjun Xie, and Fei Huang. Evolvesearch: An iterative sell-evolving search agent, 2025. URL
https://arxiv.. org/abs/ 2555/505//25..... 2505 22501
Yuxiang Zheng, Dayuan Fu, Xiangkun Hu, Xiaojic Cai, Lyumanshan Yc, Pengrui Lu, and Pengfei Liu. Deep- researcher: Scaling deep research via reinforcement learning in real-world environments. arXio preprinl arXiv:2504.0316) 2025.
Shuyan Zhou, Frank Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishok Sridhar, Xianyi Cheng, Tianyuc Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. Webarena: realistic web environment for building autonomous agents, 2024. URT. https: http///axiv..oraaa/////aa/ 13854
A Appendix
A.1
Eval Loss Curve
eval/loss
0.42
0.40
0.38
0.36
400
600
800
1000
1200
Figure 4: The evaluation loss curve of SFT.
A.2
Evaluation Prompt
Eval Prompt.
You are an agent tasked with determining the correctness an answer. Given a question, its corresponding ground truth, and an answer, you need decide whether the answer correct. it is correct, please output 'yes' otherwise output 'no'.
question: ()
ground truth: 0
answer: 0
15

---

Published in Transactions on Machine Learning Research (01/2026)
A.3 System Prompt
Prompt
You are browser interaction assistant designed to execute step-by-step browser operations efficiently and precisely to complete the user's task. You are provided with specific tasks and webpage-related information, and you need to output accurate actions to accomplish the user's task.
Here's the information you'll have:
The user's objective: This is the task you't re trying to complete. The current web page's s accessibility tree: This is a simplified representation the webpage, providing key information.
The open tabs: These are the tabs you have open.
The previous actions: There are the actions you just performed. It may be helpful to track your progress. Information already found:
Information related to the current query that has been identified in historical actions. You need to integrate and supplement this information.
The actions you can perform fall into several categorics:
Page Operation Actions:
'click [ic]] [content]": This action clicks on an element with a specific id on the webpage. 'type [id] [content] [press enter after=0) Use this type the content into the field with id. By default, the "Enter'' key is pressed after typing unless press_enter after is set to O. "hover [id] [content]': Hover over an element with id. 'press [key_ [key_comb]]: Simulates the pressing of key combination on the keyboard (e.g., Ctrl+v).
'scroll [down|up]:: Scroll the page up or down.
Tab Management Actions:
'new_tab': Open a new, empty browser tab. "tab focus [tab index]': Switch the browser's focus to a specific tab using its index.
'close_tab': Close the currently active tab.
URL Navigation Actions:
'goto [url]': Navigate to a specific URL.
"go back': Navigale to the previously viewed page.
'go fo__oorrrd': Navigate to the next page (if a previous 'go back' action was performed). Completion Action:
"stop [answer]]: Issue this action when you believe the task is complete. If the objective is find a text-based answer, provide the answer in the bracket. If you believe the task impossible to complete, provide the answer as ""NA"" in the bracket.
To be successful, it is very important to follow the following rules:
1. You should only issue an action that is valid given the current observation.
2. You should only issuc one action at a time.
3. You should follow the examples to reason step by step and then issue the next action.
4. You should refer historical actions when issue an action and try not to make repetitive actions.
tags, and there must be no output before 5. All reasoning must be inside '<think><tthink"
<<think<</thnn>>'
6. After "<think><<thinkk> only the action should be generated in the correct format, enclosed in code fences. For example:
<think>Thit button looks relevant to my goal. Clicking it should take me to the next stop.</think>
"'click [id] [content]
7. Tssue the stop action when you think you have achieved the objective. Don't generate anything after stop.
8. Always format actions correctly:
"'command [parameters]a* For example, if searching for "'death row inmates in the US* in a search field with ID '21', correctly format it as:
" "typpe [21] [death row inmates in the US] [1]*
Avoid incorrect formats that omit brackets around parameters or numeric values.
16

---

Published in Transactions on Machine Learning Research (01/2026)
9. Between <think></think>>> you need use <concusiio>><</onccuusiii to enclose the information obtained in this round that is relevant to the current query. Note that. if there is no valid information, this part is not required. The enclosed information must be directly usable to answer the original query.
A.4 Case Study
In this case of the NQ dataset, our BrowserAgent uses 3 steps to interact with the web environment and find the final answer of the given question. The ground truth of the question "Edward IV of England', but the BrowserAgent gives the final answer "King Edward IV' Obviously, these two are semantically equivalent, but cannot the EM method to match them.
Case Study
Question: Who was the father of the princes in the tower?
Step 1
Input Sequence:
System prompt:
Observation:
[331] textbox "Search 'Wikipedia'" required: False
[333] checkbox checked: false
[340] link Goo to welcome page' url: https://tigerai.ca/
[342] link "Go to the main page of 'Wikipedia' " url:
- https://tigeral /ttp://ggeaa..aaikiiieepaaaaaaaa en aam_all_mxxi__a0 2022-05/ 201__mx0__2022205/
[343] button 'Wikipedia'
[344] link 'Go to randomly selected page' url:
2022-05
History action:
History_img info:
Output Sequence:
<think>
The user's objective find out who was the father of the Princes the Tower. The current page landing page with a search box labeled 'Search 'Wikipedia" (id: 331). The most efficient way to find the answer to use the search function to look up "Princes in the Tower", this will likely lead to article that mentions their father. will type the query into the search box and press enter. </think>
Action:
type [331] [Princes the Tower] [1]
Step
Input Sequence:
System prompt:
Observation:
[742] StaticText 'Results 1-25 of 40,000 for "Princes in the Tower"
17

---

Published in Transactions on Machine Learning Research (01/2026)
[1459] link 'Princes in the Tower' url:
wttpi//eaaa.//iiiiiiiiiiiieeeeie Beradiaa____aee _a_piia_e__ll_____________llllla _a__al_maxii___ 1all_maxi_222__0 2oxi_2022-05/A/...
[790] StaticText 'from Wikipedia'
[791] StaticText 18,257 words'
[1490] link 'Shin Yokohama Prince Hotel' url:
http://tiigraa..c/vviiipppp
aall_maxi__ 2a1_mmi_i22220//AA.....
[793] StaticText '... The Shin Yokohama Prince Hotel is skyscraper in the Kohoku-ku ward of Yokohama, Japan. Completed in March 1992, it stands at 149.46 4m (490 ft) tall.
Historyy_cctee action:
type [331] [Princes in the Tower] [1]
History_ing info:
Output Sequence:
<think>
The user's objective is to find out who was the father the princes the tower. The search results page shows link titled 'Princes in the Tower," which is likely to contain relevant information about the princes and their father. Clicking this link should lead to the main article where the answer is probably found.
</think>
<conclusion.>
There is a link to the Princes in the Towerärticle, which is likely to contain information about their father. </enncusson>>
Action:
click [1459] [Princes in the Tower]
Step
Input Sequence:
System prompt:
Observation:
[3655] link 'Richard of Shrewsbury, Duke of York' url: wwrikiiedia_en_aaaee en_alllllllaaaaaaaeaa maxi 200200222222222 ...
[2285] StaticText 'These two brothers were the only sons of King'
[3656] link 'Edward IV' url:
https://tigeral httpi/ara..ca/iiii//iiiiiippppaa en all maxi 2022-05/A/...
[2287] StaticText and
[3657] link 'Elizabeth Woodville' url:
http://gggraaccakkkkkkiiiiiiaaaa en all, maxi___a_____ 2022-05/A/
History action:
type [331] [Princes in the Tower] [1]
click [1459] [Princes in the Tower]
History _nfo:
18

---

Published in Transactions on Machine Learning Research (01/2026)
There is link to the 'Princes in the Tower" article, which is likely to contain information about their father.
Output Sequence:
<think>
The static text the page states: "These two brothers were the only of King Edward IV and Elizabeth Woodville." This directly answers the question about the father of the Princes in the Tower.
</think>
<conclusion
The father of the Princes in the Tower was King Edward IV. </conclusion>>
Action:
stop [King Edward IV]
Final answer: King Edward IV
Ground truth: Edward IV of England
In this case of the HotpotQA dataset, BrowserAgent uses steps to interact with the web environment and find the final answer of the given question. The ground truth of the question is 'no', but the Browser gent gives the final answer "Skin Yard was from the U.S. (Seattle, Washington). Ostava was from Bulgaria (not the U.S.).". For this judgmental question, our agent provided a negative answer to prove that the two cities come from different countries, which equivalent to a negative answer from the ground truth.
Case Study
Question: Were the bands Skin Yard and Ostava from the U.S.?
Step
Input Sequence:
System prompt:
Observation:
[333] textbox \"Search 'Wikipedia'V required: False
[335] checkbox checked: false
[342] link 'Go to welcome page' url: https://tigerai.cal
[344] link \GGo the main page of 'Wikipedia' url:
https://tigerai.cm /ttp://gggaa..aaaaiiiiaaaaaaaaep aaclaaennaaallaaaaaaallaalllllle all m_xi mtx___a2222000//// 2022-05/
[345] button 'Wikipedia'
[346] link 'oo to randomly selected page' url:
https://tigerai. ww//random/ctttettttiiippppppppo en all maxi 2022-05
History_action: action:
History
History_iid info:
Output Sequence:
<think>
To determine whether Skin Yard and Ostava were from the U.S., I need to look up each band individually. The most efficient way to use the Wikipedia search box available on this page. will start by searching for 'Skin Yard" using the scarch textbox with id [333].
</think>
Action:
19

---

Published in Transactions on Machine Learning Research (01/2026)
type
[333] [Skin Yard]
[1]
Step
Input Sequence:
System prompt:
Observation:
[743] StaticText 'Results 1-25 of 4,000 for \"Skin Yard\"
[1155] link 'Skin Yard' url:
http:/tigeraa.ccawikiiadda.. _a_pila_____l__________________a all maxi max1__002-00/A/Sktnn Yard
[750] StaticText 'Skin Yard Skin Yard an American grunge band from Seattle, Washington, who were active from 1985 to 1992. The group never gained mainstream audience, but were an influence on several of their grunge
contemporaries, including Soundgarden, Screaming Trees, and Green River. Skin Yard Publicity photo 1988, feat. Jack Endino, Ben McMillan, and Daniel House Background information Origin Seattle, Washington, U.S. Genres Grunge[1] alternative metal Years active 1985-1992 Labels Cruz, Toxic Shock......
History action:
type [333] [Skin Yard] [1]
History info:
Output Sequence:
<think>>
We have clear evidence from the static text that Skin Yard was an American grunge band from Scattle, Washington, U.S. This answers the first part the question. Now, we need to check Ostava was from the U.S. The current page focused on Skin Yard, so should search for Ostava next. The search box has id [1133], so will type 'Ostava" into it and search.
< </think>
<conclusion
Skin Yard was from Seattle, Washington, U.S.
</conclusion>>
Action:
type [1133] [Ostava] [1]
Step 3
Input Sequence:
System prompt:
Observation:
[1455] StaticText 'Results 1-24 of 24 for \"Ostava\"'
[1630] link 'Ostava' url:
20

---

Published in Transactions on Machine Learning Research (01/2026)
[1462] StaticText 'Ostava Ostava is an alternative rock band from Bulgaria. It was formed in 1991, and released a long-playing record in 2000. Ostava Background information Origin Gabrovo, Bulgaria Genres Alternative rock Years active 1991-present Labels Riva Sound Stars records Stain Studio Virginia
Records/Universal Members Svilen Noev Georgi Georgiev Daniel Petrov Boyan Petkov Mihail Shishkov Alexander Marburg History Formation and early years Fender Jazz Bass played by Boyan Petkov Ostava was formed in 1991
History action:
type [333] [Skin Yard] [1]
type [1133] [Ostava] [1]
History info:
Skin Yard was from Seattle, Washington, U.S.
Output Sequence:
<hhink>
We already know from HISTORY info that Skin Yard was from Seattle, Washington, U.S. Now, from the current observation, we see that Ostava is described as 'an altornative rock band from Bulgaria. was formed in 1991...' and its origin listed as Gabrovo, Bulgaria. Therefore, Ostava not from the U.S., but from Bulgaria.
</think>
<conclusion>>
Skin Yard was from the U.S. (Seattle, Washington). Oslava was from Bulgaria (not the U.S.). </conclusion>
Action:
stop [Skin Yard was from the U.S. (Seattle, Washington). Ostava was from Bulgaria (not the U.
Final answer: Skin Yard was from the U.S. (Seattle, Washington). Ostava was from Bulgaria (not the U.S.).
Ground truth: no
A.5 filtering strategy and data ratio
We used string matching filter out RFT data that did not conform to our expected format. To preserve the original data distribution and mitigate the calastrophic forgetting of answer formatting learned during the SFT stage, we randomly sampled 80% of the SFT data to include in the final dataset used for the second- stage training. To ensure consistency in data volume and proportion between the two stages, we randomly selected 400 NQ samples and 673 HotpotQA samples from the filtered Original RFT data, resulting in a total of 1,073 representative reasoning examples. By combining these 1,073 samples with the 80% subset of SFT data, we obtained the final dataset used for RFT training.
A.6 Experimental Setup
During the training phase, we adopt the 5.3K synthesized data for SFT and RFT. Both training and evaluation were conducted on NVIDIA A100 GPUs. The SFT stage was performed for 2 epochs with learning rate of 56-6, while the RFT stage was also trained for epochs, using a lower lcarning rate of 1c-6. Model evaluation was conducted on the test or validation sets of six publicly available question answering datasets, covering diverse range of tasks systematically assess the model's generalization ability in
21

---

Published in Transactions on Machine Learning Research (01/2026)
both in-domain and out-of-domain settings. The evaluation metrics include EM and LLM-based judgment, providing a comprehensive reflection of the model's accuracy and real-world applicability.
A.7 Experimental Results
A.7.1
Test Step Scaling
Table 4: Results of the ablation study on BrowserAgent-Rii
General QA
Multi-Hop QA
Avg.
Steps
NQ
PopQA
HotpoiQA
2wiki
Musique
Bamboogle
IND
OOD
IND
OOD
OOD
OOD
6
0.361
0.399
0.402
0.409
0.133
0.352
0.343
15
0.384
0.404
0.419
0.449
0.145
0.480
0.380
30
0.390
0.465
0.465
0.457
0.161
0.504
0.407
Table 5: Average number of steps in the ablation study on BrowserAgent-RFT.
General QA
Multi-Hop QA
Avg.
Steps
NQ
PopQA
HotpotQA
2wiki
Musique
Bamboogle
IND
OOD
IND
OOD
OOD
OOD
6
3.039
2.799
3.442
4.022
4.128
3.909
3.557
15
3.516
3.248
3.979
4.999
4.648
4.400
4.132
30
3.933
3.856
3.856
4.945
4.638
4.492
4.287
A.7.2 TriviaQA Results
As shown in the Table G there is a significant discrepancy between the FM scores and LLM-judge evaluation scores for our two models on the TriviaQA datasct. We attribute this gap to the absence of format-spocific supervision signals in our training process, which leads to answers that are factually correct but fail to satisfy exact match criteria due to formatting issues. For detailed examples, please refer the case study in the Appendix A.4
Table 6: TriviaQA results.
Model
TriviaQA
Browseragent-SFT (EM)
0.206
Browseragent-RF" (EM)
0.206
Browseragent-SFT (LLM-judge)
0.598
Browseragent-RIFT (LLM-judge)
0.600
22


Reviews:
[{"review_id": "r8vMVW421I", "reviewer": "Reviewer_u76V", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "## Summary\n\nThe paper presents BrowserAgent, a browser native web agent that answers open domain QA by interacting with web pages through Playwright actions.\n\n## Strengths\n\n- The paper clearly motivates the move from offline HTML parsing to in browser actions for web agents. I agree that this setting is motivated by real life applications.\n- The training pipeline uses SFT plus rejection sampling on a modest number of trajectories and does not require RL training\n- The model shows strong empirical gains across several QA benchmarks.\n- The Ray for Playwright infra for running many browser instances in parallel is practically useful and will be helpful for the community.\n\n## Weaknesses/Questions\n\n\n- The paper uses the terms IND and OOD in Table 2, but I could not find where these terms are defined, or what these terms mean for each dataset?\n- The paper states that exact match and LLM-Judge are the two evaluation metrics, but for the baselines in Table 2, especially the Search R1 variants, it is not clear which metrics they are evaluated on. Also, are the numbers taken from prior work or reproduced by the authors?\n- I am a bit confused about the difference between Browseragent-SFT and Browseragent-RFT, since the authors mentioned in Section 2.4 that they propose a two-stage training framework. Does  Browseragent-SFT stand for only step 1 (SFT) and Browseragent-RFT stand for both steps (SFT+RFT)?\n- Are you going to open-source the Ray-parallelized orchestration for Playwright?", "claims_and_evidence_yn": "Yes", "claims_explanation": "Mostly yes. The main claims (a browser native agent with a small action set and SFT plus RFT training can outperform strong search based baselines on Wikipedia QA) are supported by the reported experimental results. \n\nThe broader claims about real-world web interaction feel a bit ahead of the evidence, since experiments are limited to QA in the context of Wikipedia. Clarifying this scope would strengthen the paper.", "audience_yn": "Yes", "audience_explanation": "Yes. I think researchers working on web agents, RAG, and RL for agentic applications would be interested. The Ray-parallelized orchestration for Playwright will be a useful resource for the community.", "requested_changes": "See Weaknesses/Questions.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "BpPW5ivptr", "reviewer": "Reviewer_WnG1", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes BrowserAgent, a browser-native web agent that operates directly on dynamic webpages through a minimal set of human-aligned actions (click, type, scroll, hover, and structured navigation). Unlike prior systems that rely heavily on search tools, scripted parsers, or static page summaries, BrowserAgent grounds the agent entirely in real browser states, accessed through the accessibility tree. The authors introduce a two-stage post-training pipeline—SFT followed by RFT—that enables an LLM to acquire stable long-horizon browsing behavior while using only 5.3K curated trajectories. A lightweight memory mechanism allows the agent to accumulate intermediate conclusions in a structured form, improving multi-hop reasoning without excessive context growth. The system is supported by a parallelized Playwright-based environment that enables efficient data collection and evaluation.\n\n**Strengths include:** (1) a clean and browser-native formulation of web-agent actions, (2) an efficient and practical post-training strategy, (3) a simple yet effective memory design enabling long-horizon reasoning, and (4) thorough empirical and ablation analyses.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The paper’s main claims are supported by consistent empirical evidence. BrowserAgent outperforms Search-R1-Instruct across all six evaluated QA benchmarks, with especially large gains on multi-hop datasets such as HotpotQA and 2Wiki (often exceeding 10% absolute improvement). These results directly validate the claim that grounding the agent in real browser states enables more reliable long-horizon reasoning. The ablation studies also substantiate the paper’s design choices: removing the memory module leads to 8–12% performance drops on multi-hop tasks, and omitting RFT causes clear degradation across datasets. Increasing the step budget from 6 to 30 yields monotonic improvements, supporting the claim that the system can maintain stable multi-step trajectories.", "audience_yn": "Yes", "audience_explanation": "This paper addresses a timely and active research direction: enabling LLMs to function as web agents that interact with real, dynamic webpages rather than static summaries or search results. BrowserAgent offers a browser-native framework, a lightweight SFT+RFT training pipeline, and a structured memory mechanism—components. The significant improvements over Search-R1-Instruct on multi-hop QA benchmarks further demonstrate that the proposed approach yields practical gains, making the findings relevant to researchers developing agentic LLM systems and training methodologies.", "requested_changes": "**1. Inconsistency in Evaluation Metric vs. RFT Data Filtering (Section 2.4 & Section 3.2):** In Section 3.2 (Evaluation Metrics), the authors explicitly state that \"EM often underestimates performance when answers are factually correct... but differ in surface form,\" and thus introduce LLM-based judgment as a complementary metric. However, in Section 2.4 (Stage 2: RFT with Reasoning Path Selection), the methodology relies exclusively on EM (Exact Match) to filter the reasoning paths for Rejection Fine-Tuning.\nIf EM is acknowledged as a flawed metric for identifying correct answers in this domain, using it as the sole gatekeeper for generating RFT training data may potentially discard some of the high-quality reasoning trajectories that are semantically correct but not exact matches.\n\n**2. Validity of \"Longest Trajectory\" Selection Strategy in RFT (Section 2.4):** In Section 2.4, the paper describes the RFT process, stating: \"From these instances, we choose the correct answer that contains the largest number of reasoning steps... as more reasoning steps indicate deeper model thinking.\" This assumption is highly debatable, especially from efficiency perspective. A longer trajectory may not necessarily imply \"deeper thinking\". For example, it may contain \"overthinking\" steps or useless actions.\n\n**3. Validation and Statistics of Action Usage (Section 3.5 & Table 1):** Although Table 1 defines a broad set of \"Human-Inspired\" actions (e.g., hover, tab_focus, scroll), it remains unclear whether the trained model actually leverages this full spectrum or converges to a limited subset like type and click. To substantiate the claim of \"human-inspired browsing,\" please provide a statistical breakdown or histogram of action frequency across test trajectories, and discuss whether specific actions like scroll or tab_management positively contribute to success rates or are merely superficial additions.", "broader_impact_concerns": "No concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "jQ0rp0PyzJ", "reviewer": "Reviewer_JBgb", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces BrowserAgent, a browser-native agent that performs human-like actions (scroll, click, type, tab operations) via Playwright automation library. Unlike prior approaches (like Search-R1) that rely on external tools to convert dynamic web pages into static text summaries, BrowserAgent operates directly on the live Accessibility Tree. The authors propose a scalable Ray-based orchestration system (50+ episodes/min), a two-stage SFT, RFT training pipeline over 5.3K trajectories, and a lightweight explicit-memory mechanism for multi-step reasoning. \nEmpirically, BrowserAgent convincingly outperforms Search-R1 on multi-hop tasks by ~20% (Figure 1 and Table 2), especially in long-horizon reasoning scenarios. They augment ReAct with an explicit memory mechanism to reduce bloated context. Ablations support the design decisions.\nThe paper is clearly written and technically sound, and the engineering contribution is substantial.\n\n**Strengths:**\n\n1. **Clear conceptual contribution**\nShifting from HTML summarization to direct browser action is a meaningful design decision. The paper argues convincingly that current pipelines relying on external summarization limit depth of interaction and increase cost. BrowserAgent’s results support this motivation.\n\n2. **Strong engineering contribution**\nThe work presents a scalable Playwright-based interaction system using Ray in section 2.3 , which significantly increases throughput (over 50 episodes/min). This effectively addresses a known bottleneck in browser-native agent training. \n\n3. **Effective memory design**\nThe structured `<conclusion>`-based memory mechanism stabilizes long-horizon reasoning while reducing context size. Ablation results (Table 3) show clear benefits.\n\n4. **Strong empirical performance**\nBrowserAgent achieves consistent improvements across both general QA and multi-hop QA. The gains over Search-R1 are significant, especially considering smaller training data size, simpler training pipeline, no reliance on external summarizers or HTML parsing.\n\n**Weaknesses:**\n\n1. The major gap of this paper is that all experiments are conducted over a local Kiwix mirror of Wikipedia using Playwright’s accessibility tree (Section 2.3). While this provides a controlled setting, it fails to represent the noise, dynamic content (JavaScript, pop-ups, ads), and structural diversity of the open web. Thus, the claim of solving “complex web tasks” (Abstract, Introduction) is not supported by the empirical evidence. The evaluation setup reflects *retrieval QA over static pages*, not general-purpose web navigation. Because BrowserAgent operates on text-only accessibility trees and is trained exclusively on Wikipedia, it is unclear whether the method extends to real websites such as Amazon, [Booking.com](http://booking.com/), or news sites that rely heavily on dynamic content, canvas elements, or unlabeled icons.\n\n2. The paper ignores established web agent benchmarks such as WebArena, BrowserGym, Mind2Web, and VisualWebArena, despite citing them. The authors justify avoiding WebArena because it \"cannot be efficiently parallelized\". While engineering constraints are valid concerns, omitting the standard benchmark in the field makes it impossible to assess the model's generalization capabilities. Evaluating solely on QA datasets (HotpotQA, NQ)  ignores the navigational complexity that defines web agents. As a result, the reader cannot assess BrowserAgent’s generalization beyond Wikipedia-based QA tasks.\n\n3. In the RFT stage Section 2.4, the authors select the correct answer that utilizes the *largest* number of reasoning steps. The justification is that more steps indicate \"deeper model thinking\". However, in web navigation:\n- longer trajectories typically indicate inefficiency,\n- prior RL work penalizes excess steps to encourage correctness and speed (e.g., WebGPT, WebArena agents),\n- longer step sequences may reflect unnecessary scrolling or redundant actions.\nThis heuristic risks training the agent toward verbosity or circuitous navigation. An ablation comparing “shortest correct trajectory” vs. “longest correct trajectory” is necessary to justify this design choice.\n\n4. In the System Prompt (Appendix A.3), this appears to be implemented simply by instructing the model to output a `<conclusion>` tag, which is then appended to the context history. Is there any pruning mechanism for this memory? If the agent runs for maximum 30 steps, does the accumulation of `<conclusion>` tags eventually crowd out the observation window? Can the  contradictory or incorrect early conclusions can poison later reasoning? \nA more detailed analysis of memory stability, error propagation, and scalability is helpful.\n\n5. Because the agent relies on deterministic element IDs from accessibility trees, it is unclear how robust it is to:noisy accessibility annotations ,unlabeled interactive elements, changes in DOM structure, differences across websites.\nThis is a core requirement for web agents, and the paper does not investigate it.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The evidence is accurate and internally consistent for the environment studied (Kiwix-based Wikipedia). The improvements over Search-R1 and the benefits of the memory mechanism are well supported by the results in Table 2 and Table 3. The orchestration throughput is documented clearly in Section 2.3.\n\nHowever, the evidence does not fully support broader claims about “complex web tasks” or real-world web navigation, since all evaluation is conducted on static Wikipedia pages. The claims are valid within the scope of the controlled environment, but generalization to real websites is not demonstrated.", "audience_yn": "Yes", "audience_explanation": "The work is relevant to researchers in LLM agents, web-based reasoning, and scalable browser automation. Improving browser-native agents is an active area of interest, especially following recent progress in deep research agents. The paper offers a great engineering contribution for parallelizing playwright infrastructure and provides a compelling alternative paradigm to HTML summarization pipelines.", "requested_changes": "**Critical (required for acceptance):**\n1. Evaluate on at least one standard web-agent benchmark (e.g., WebArena, Mind2Web, BrowserGym, or VisualWebArena).\nEven partial or qualitative evaluation would strengthen claims of generalization beyond Wikipedia.\n2. Provide an ablation comparing “shortest correct trajectory” vs. “longest correct trajectory,” and analyze how the choice affects navigation efficiency.\n3. Please provide some more details on the analysis of the memory mechanism. Discuss pruning, error correction, and failure modes. Show whether accumulated <conclusion> tags affect performance on long trajectories.\n4. Discuss robustness to real-world accessibility trees. Include experiments or qualitative evidence showing BrowserAgent’s behavior on dynamic or poorly annotated pages.\n\n**Important (would significantly strengthen the paper):**\n\n5. Add qualitative examples on real websites (even if performance is poor).\n6. Discuss the limitations of using only Kiwix/Wikipedia as the environment.\n7. Provide more details on Ray parallelization and scaling beyond a single 32-core machine.", "broader_impact_concerns": "No major ethical concerns beyond standard issues associated with autonomous browsing agents.However, as browser-native agents can interact with live websites, potential misuse for automated scraping, credential entry, or unintended actions should be discussed. A very short Broader Impact Statement addressing responsible deployment and safety considerations would be nice.", "additional_comments": "- The paper is clearly written and easy to follow.\n\n- Figures and tables are informative; Figure 3's workflow is especially helpful.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)