R-WoM: Retrieval-augmented World Model For Computer-use Agents
Authors: name: Kai Mei; name: Jiang Guo; name: Shuaichen Chang; name: Mingwen Dong; name: Dongkyu Lee; name: Xing Niu; name: Jiarong Jiang
Published: 2025-10-13 19:52:04

Abstract: Large Language Models (LLMs) can serve as world models to enhance agent
decision-making in digital environments by simulating future states and
predicting action outcomes, potentially eliminating costly trial-and-error
exploration. However, this capability is fundamentally limited by LLMs'
tendency toward hallucination and their reliance on static training knowledge,
which can lead to compounding errors that inhibit long-horizon simulations. To
systematically investigate whether LLMs are appropriate for world modeling, we
probe two core capabilities of world models--future state prediction and reward
estimation--through three tasks: next-state identification, full-procedure
planning alignment, and milestone transition recognition. Our analysis shows
that while LLMs effectively capture immediate next states and identify
meaningful state transitions, their performance rapidly degrades in
full-procedure planning. This highlights LLMs' limitations in reliably modeling
environment dynamics over long horizons. To address these limitations, we
propose the Retrieval-augmented World Model (R-WoM), which grounds LLM
simulations by incorporating factual, up-to-date knowledge retrieved from
external tutorials. Experiments show that R-WoM achieves substantial
improvements of up to 25.3% (OSWorld) and 18.1% (WebArena) compared to
baselines, with particular advantages in longer-horizon simulations.