L ite W eb A gent: The Open-Source Suite for VLM -Based Web-Agent Applications (2025)

Abstract: We introduce LiteWebAgent, an open-source suite for VLM-based web agent applications. Our framework addresses a critical gap in the web agent ecosystem with a production-ready solution that combines minimal serverless backend configuration, intuitive user and browser interfaces, and extensible research capabilities in agent planning, memory, and tree search. For the core LiteWebAgent agent framework, we implemented a simple yet effective baseline using recursive function calling, providing with decoupled action generation and action grounding. In addition, we integrate advanced research components such as agent planning, agent workflow memory, and tree search in a modular and extensible manner. We then integrate the LiteWebAgent agent framework with frontend and backend as deployed systems in two formats: (1) a production Vercel-based web application, which provides users with an agent-controlled remote browser, (2) a Chrome extension leveraging LiteWebAgent’s API to control an existing Chrome browser via CDP (Chrome DevTools Protocol). The LiteWebAgent framework is available at https://github.com/PathOnAI/LiteWebAgent, with deployed frontend at https://lite-web-agent.vercel.app/.

## introduction and related work recent advancements in large language models (llms) and vision-language models (vlms) have revolutionized web browser automation. by late 2023, models like gpt-4v demonstrated the ability to handle real-world tasks on complex platforms such as reddit and gitlab @xcite . this marked a shift from basic ui commands on toy web pages @xcite to sophisticated automation, driving research into vlm-powered agents for browser and device control. companies like openai, anthropic, adept, google, and apple are actively developing autonomous agents, recognizing their transformative potential. while significant progress has been made, automation tasks can generally be categorized into two main approaches: web agents and device control agents. the following section outlines their differences and respective strengths. ## litewebagent agent framework in this section, we present the design of the web agent framework implemented in the open-source repository litewebagent @xcite . specifically, the framework decouples action generation from action grounding, supports multiple versions of agent planning, and integrates agent memory into the planning process. this design enables effective management of long-horizon tasks that require both planning and error recovery. figure 1 illustrates an overview of the web agent's workflow. moreover, we explain how the agent framework is extended with tree search to explore multiple trajectories and balance exploitation with exploration. ## conclusion and future work litewebagent: the open-source suite for vlm-based web-agent applications offers a robust toolkit to address key gaps in the web agent ecosystem. it features: (1) an extensible core agent framework with simple yet effective baseline implementations, offering a scalable foundation for integrating new research such as agent workflows, memory management, planning, and tree search, while making it easy for developers to add additional tools beyond web browsing; (2) asynchronous apis that integrate seamlessly with fastapi, supporting serverless deployment on platforms like vercel; (3) a flexible user interface with configuration options, a chat interface, voice integration, and task execution visualization; and (4) two types of browser environments correspond to the deployed systems: interactions with existing browsers via the chrome devtools protocol (cdp) and a production-ready, vercel-based full-stack application -an open-source, multion-style service using a remote browser. in the future, our objectives include: (1) extending the demo to build the first productionready web agent with tree search demo with a userfriendly interface and intuitive visualization, (2) integrating litewebagent into multi-agent frame-works such as @xcite to enable multiagent frameworks with web browsing capabilities, (3) adding an evaluation module including new metrics to more comprehensively evaluate web agent performance.