Web Agents with World Models: Learning and Leveraging Environment
  Dynamics in Web Navigation
Authors: name: Hyungjoo Chae; name: Namyoung Kim; name: Kai Tzu-iunn Ong; name: Minju Gwak; name: Gwanwoo Song; name: Jihoon Kim; name: Sunghwan Kim; name: Dongha Lee; name: Jinyoung Yeo
Published: 2024-10-17 05:37:00

Abstract: Large language models (LLMs) have recently gained much attention in building
autonomous agents. However, the performance of current LLM-based web agents in
long-horizon tasks is far from optimal, often yielding errors such as
repeatedly buying a non-refundable flight ticket. By contrast, humans can avoid
such an irreversible mistake, as we have an awareness of the potential outcomes
(e.g., losing money) of our actions, also known as the "world model". Motivated
by this, our study first starts with preliminary analyses, confirming the
absence of world models in current LLMs (e.g., GPT-4o, Claude-3.5-Sonnet,
etc.). Then, we present a World-model-augmented (WMA) web agent, which
simulates the outcomes of its actions for better decision-making. To overcome
the challenges in training LLMs as world models predicting next observations,
such as repeated elements across observations and long HTML inputs, we propose
a transition-focused observation abstraction, where the prediction objectives
are free-form natural language descriptions exclusively highlighting important
state differences between time steps. Experiments on WebArena and Mind2Web show
that our world models improve agents' policy selection without training and
demonstrate our agents' cost- and time-efficiency compared to recent
tree-search-based agents.