W eb UIB ench: A Comprehensive Benchmark for Evaluating Multimodal Large Language Models in W eb UI -to-Code (2025)

Abstract: With the rapid advancement of Generative AI technology, Multimodal Large Language Models(MLLMs) have the potential to act as AI software engineers capable of executing complex web application development. Considering that the model requires a confluence of multidimensional sub-capabilities to address the challenges of various development phases, constructing a multi-view evaluation framework is crucial for accurately guiding the enhancement of development efficiency. However, existing benchmarks usually fail to provide an assessment of sub-capabilities and focus solely on webpage generation outcomes. In this work, we draw inspiration from the principles of software engineering and further propose WebUIBench, a benchmark systematically designed to evaluate MLLMs in four key areas: WebUI Perception, HTML Programming, WebUI-HTML Understanding, and WebUI-to-Code. WebUIBench comprises 21K high-quality question-answer pairs derived from over 0.7K real-world websites. The extensive evaluation of 29 mainstream MLLMs uncovers the skill characteristics and various weakness that models encountered during the development process.

## introduction the emergence of large language models(llms) has rapidly reshaped the landscape of software engineering. ai code generation @xcite @xcite evolves from assisting developers to independently completing the entire development lifecycle (i.e., ai software engineer). automatic website development is a challenging and widely discussed multimodal code generation scenario @xcite @xcite : multimodal large language models(mllms) are required to generate front-end code projects based on user-provided webui images (webui-to-code). recent works @xcite @xcite have evaluated mllms and reached a consensus that mllms struggle to generate complex websites, revealing a significant gap between solutions and practical applications. therefore, it is essential to identify the challenges across various development stages and evaluate the corresponding sub-capabilities of models. however, current benchmarks @xcite typically focus on assessing the output quality of generated website (e.g., webpage elements and layout) and mostly lack evaluation for subcapabilities. to address this issue, @xcite propose webpage understanding benchmark, but they are restricted to only one type of sub-capability and limited to the dataset quality annotated by llms. inspired by the main activities of software engineering @xcite , we initially pro-pose a taxonomy for the capability evaluation of sub-capability as depicted in figure 1 . our core idea is to align the evaluation criteria of models' capabilities with the task requirements of software engineering. to this end, we propose the following sub-capability evaluation dimensions: (i) webui perception: webui images serve as visual carriers of development requirements. the fundamental skill of the model is to accurately perceive the visual semantic information in the webpage, including both text and images. (ii) html programming: during the software construction phase, the model's knowledge reservoir and programming skills in front-end code are essential for assisting or substituting developers for efficient development. and (iii) webui-html understanding: post-software development, code testing and adjustments are necessary to ensure requirement accuracy. this necessitates the model's ability to perform cross-modality reasoning between design images and code functionalities. furthermore, we draw from current leading mllms evaluation benchmarks @xcite @xcite to design multiple subtasks for each evaluation dimension, tailored to the characteristics of web data. in summary, our main contributions are three-fold: the observed positive correlation between sub-capabilities and webui-to-code performance validates our evaluation approach. it also reveals that the primary challenge for current mllms is to enhance and balance subcapabilities across different dimensions. ## taxonomy of evaluation inspired by web application development, our benchmark evaluates webui-to-code(task9) capability, and three essential sub-capabilities: we-bui perception, html programming, and webui-html understanding. for webui-to-code, we provide two types of webpage: full webpage and webpage slice. for sub-capability evaluation, we designed various sub-tasks as follows: ## conclusion in this study, we introduce webuibench, a largescale and comprehensive benchmark designed to evaluate the webui-to-code capabilities of multimodal large language models (mllms). we-buibench comprises over 21k question-answer pairs derived from more than 0.7k real-world websites, encompassing 9 distinct subtasks. we conducted extensive experiments on 7 state-of-theart closed-source and 22 prominent open-source mllms. our key findings highlight the models' deficiencies in webpage generation tasks across various dimensions, including cross-modality reasoning, element localization, and webpage layout generation. this benchmark provides critical insights and guidance for future research aimed at improving webpage generation performance.