WebWalker: Benchmarking LLMs in Web Traversal
Authors: name: Jialong Wu; name: Wenbiao Yin; name: Yong Jiang; name: Zhenglin Wang; name: Zekun Xi; name: Runnan Fang; name: Deyu Zhou; name: Pengjun Xie; name: Fei Huang
Published: 2025-01-13 18:58:07

Abstract: Retrieval-augmented generation (RAG) demonstrates remarkable performance
across tasks in open-domain question-answering. However, traditional search
engines may retrieve shallow content, limiting the ability of LLMs to handle
complex, multi-layered information. To address it, we introduce WebWalkerQA, a
benchmark designed to assess the ability of LLMs to perform web traversal. It
evaluates the capacity of LLMs to traverse a website's subpages to extract
high-quality data systematically. We propose WebWalker, which is a multi-agent
framework that mimics human-like web navigation through an explore-critic
paradigm. Extensive experimental results show that WebWalkerQA is challenging
and demonstrates the effectiveness of RAG combined with WebWalker, through the
horizontal and vertical integration in real-world scenarios.