IWR-Bench: Can LVLMs reconstruct interactive webpage from a user
  interaction video?
Authors: name: Yang Chen; name: Minghao Liu; name: Yufan Shen; name: Yunwen Li; name: Tianyuan Huang; name: Xinyu Fang; name: Tianyu Zheng; name: Wenxuan Huang; name: Cheng Yang; name: Daocheng Fu; name: Jianbiao Mei; name: Rong Wu; name: Licheng Wen; name: Xuemeng Yang; name: Song Mao; name: Qunshu Lin; name: Zhi Yu; name: Yongliang Shen; name: Yu Qiao; name: Botian Shi
Published: 2025-09-29 12:38:06

Abstract: The webpage-to-code task requires models to understand visual representations
of webpages and generate corresponding code. However, existing benchmarks
primarily focus on static screenshot-to-code tasks, thereby overlooking the
dynamic interactions fundamental to real-world web applications. To address
this limitation, this paper introduces IWR-Bench, a novel benchmark for
evaluating the capabilities of Large Vision-Language Models (LVLMs) in
interactive webpage reconstruction from video. IWR-Bench comprises 113
meticulously curated tasks from 100 real-world websites, with 1,001 actions and
featuring diverse interaction complexities (e.g., web games), visual styles,
and domains. Aligning with standard web development practices, each task
includes not only user interaction videos but also all crawled static assets
(e.g., images, videos). This benchmark evaluates models on two fundamental
challenges: comprehensive multi-modal reasoning to infer interaction logic from
video and assets, and advanced code generation to translate this logic into
functional code. An agent-as-a-judge framework with a comprehensive metric
system automatically assesses the functional correctness and visual fidelity of
generated webpages. Extensive experiments on 28 LVLMs reveal a significant
challenge: the best model achieves an overall score of only 36.35%, as
functional correctness (24.39% IFS) lags significantly behind visual fidelity
(64.25% VFS). These results highlight critical limitations in current models'
ability to reason about temporal dynamics and synthesize event-driven logic,
establishing IWR-Bench as a challenging frontier for vision-language research.
The benchmark and evaluation code will be made publicly available. Code is
available at https://github.com/L-O-I/IWR-Bench.