Code Aesthetics with Agentic Reward Feedback
Authors: Bang Xiao
Lingjie Jiang
Shaohan Huang
Tengchao Lv
Yupan Huang
Xun Wu
Lei Cui
Furu Wei
Venue: ICLR 2026 Poster (2026)
Keywords: Large Language Model
Code Aesthetics
Reinforcement Learning
Primary area: reinforcement learning
TL;DR: We introduce "agentic reward feedback", a new reward framework for training LLMs in code aesthetic tasks, and release related dataset and benchmark.

Abstract: Large Language Models (LLMs) have become valuable assistants for developers in code-related tasks. While LLMs excel at traditional programming tasks such as code generation and bug fixing, they struggle with visually-oriented coding tasks, often producing suboptimal aesthetics. In this paper, we introduce a new pipeline to enhance the aesthetic quality of LLM-generated code. We first construct AesCode-358K, a large-scale instruction-tuning dataset focused on code aesthetics. Next, we propose agentic reward feedback, a multi-agent system that evaluates executability, static aesthetics, and interactive aesthetics. Building on this, we develop GRPO-AR, which integrates these signals into the GRPO algorithm for joint optimization of functionality and code aesthetics. Finally, we develop OpenDesign, a benchmark for assessing code aesthetics. Experimental results show that combining supervised fine-tuning on AesCode-358K with reinforcement learning using agentic reward feedback significantly improves performance on OpenDesign and enhances results on existing benchmarks such as PandasPlotBench. Notably, our AesCoder-4B surpasses GPT-4o and GPT-4.1, and achieves performance comparable to large open-source models with 480B–685B parameters, underscoring the effectiveness of our approach.

Published as a conference paper at ICLR 2026
CODE AESTHETICS WITH AGENTIC REWARD FEED- BACK
Bang Xiao Lingjie Jiang!.3-
Shaohan Huang
Tengchao Lv
Yupan Huang' Xun Wu² Lei Cui
Furu Wei
Microsoft Research Asia
2 Zhiyuan College, Shanghai Jiao Tong University
3 Peking University
sherlock bangês edu. en (shaohanh, fuwei (twei)emicroson wwwieeemm.cccccooooooooooooooo
Linggiejianggstu.pp edu. edu. cn
ABSTRACT
Large Language Models (LLMs) have become valuable assistants for developers in code-related tasks. While LLMs excel at traditional programming tasks such as code generation and bug fixing, they struggle with visually-oriented coding tasks, often producing suboptimal aesthetics. In this paper, introduce new pipeline to enhance the aesthetic quality of L.LM-generater code. We first con- struct AesCode-358K, large-scale instruction-tunning dataset focused on code aesthetics. Next, we propose agentic reward feedback, a multi-agent system that evaluates executability, static aesthetics, and interactive aesthetics. Building this, we develop GRPO-AR, which integrates these signals into the GRPO al- gorithm for joint optimization of functionality and code aesthetics. Finally, we develop OpenDesign, a benchmark for assessing code aesthetics. Experimental results show that combining supervised fine-tuning on AesCode-358K with rein- forcement learning using agentic reward feedback significantly improves perfor- mance on OpenDesign and enhances results on existing benchmarks such as Pan- dasPlotBench. Notably, our 4esCoder-4B surpasses GPT-4o and GPT-4.1, and achieves performance comparable to large spencsource models with parameters, underscoring the effectiveness of our approach.
INTRODUCTION
LLMs have become powerful assistants in our daily lives, helping polish writing, refine code, and access knowledge (Team] 2025 DeepSeek-Al et al. 2025 OpenAl 2025). Recently, coding LLMs have achieved great success in various code related fields, such as code completion, bug fixing, and 2025 Guo et al., 2024). While LLMs have demonstrated remark- coding tasks, they remain inadequate in visually-oriented
software paaeeerring(Athroppp
able capabilities in single-text-modalee tasks such as chart generation and webpage design, leading to poor visual outcomes like overlap- ping elements, inconsistent color schemes, and disorganized structures. Consequently, the aesthetic dimension of LLMs remains undercxplored area.
In this paper, we focus on assessing and improving LLMs ability in visually coding tasks, which refer programming tasks in which the correctness quality of the code is inherently tied to its visual output. Typical examples include tasks that generate or manipulate visual artifacts such web pages (HTML/CSS), plots and charts (e.g., Matplotlib (Hunter, 2007), Seaborn (Waskom, 2021), Plotly (Inc.] 2015)), or graphical scenes (e.g., Python Turtle). Unlike purely algorithmic coding tasks, these tasks require the model to reason about visual structure, spatial layout, and aesthetic consistency, in addition to syntactic or functional correctness. For the visually-orient coding tasks, natural question arises: do LLMs possess any awareness of the aesthetics of their own code? In other words, do they have sense of aesthetics?
"Equal contribution. Project pape: https: pectps:/bbaggae giub.. wttp:///oooeeeeeeeeeeeeeeeeeeeee
Cooreepponding author.

---

Published as a conference paper at ICLR 2026
Building these insights, we propose the code aesthetics concept, which captures the aesthetic appeal of visually-oriented code. Currently, reward methods for training coding LLMs often focus on single textual modality, such code executability and result correctness (Gehring et al.] [024] Fu et al. 2023 Le et al., 2022 Dai et al. 2025). These methods have significant limitations when applied to code aesthetics tasks, as they fail to assess visual aesthetics and are unable interact with elements like webpages, making them ineffective as reward sources. To address this challenge, we propose agentic reward feedback, a new reward framework consisting of three agents, (i) execution agent, which checks the code executability, (ii) static aesthetics agent, which assesses the aesthetics based on an image of code execution result, and (iii) interactive aesthetics agent, which is specified to evaluate the function of webpages while interacting with web elements. When receiving a raw model output, the execution agent will try to extract the code blocks from the output and check its executability. If passed, static aesthetics agent and interactive aesthetics agent will then run in parallel to assess the static and interactive aesthetics perspectives respectively. The core idea is simple: adopting a multi-agent system to provide comprehensive and systematic reward feedback from textual, visual, and interactive perspectives, thus giving a comprehensive feedback to better align the sense of aesthetics of model with human advanced models. This approach addresses key limitation of most open-source coding LLMs, which are confined to a single textual modality and thus lack awareness of the visual rendering of their code.
To achieve this goal, we first build a large-scale supervised instruction tuning dataset AesCode-358K
of two major code aesthetics tasks: Python-basee plot generation and webpage design. Given the ab- sense of existing benchmarks for evaluating webpage aesthetics, we also construct the OpenDesign benchmark, which consists of 840 real webpage design cases, to evaluate the aesthetics of webpage from both visual (static) and interactive aesthetics using LLM-as-a-judde (Zheng et al. 2023 Gu al. 2025) method. Consequently, we perform reinforcement learning using GRPO (Shao et al. 2024 algorithm combined with our Agentic Reward framework (GRPO-AR) to train two models with different parameter Aaseser 4B and AesCoder 7B. After supervised fine-tuning on the AesCode-3500 358K dataset and reinforcement learning with GRPO-AR, our models achieve signifi- cant improvement in PandasPlotBenchl Ban.aaoooBeecchGalaayaaaeg et al. 2025) and OpenDesign, showcasing the effectiveness of the AesCode-3588 dataset and GRPO-AR method.
The key contributions can be summarized as follows:
We introduce the concept of code aesthetics and investigate whether LLM-gemerater code demonstrates its own design aesthetics.
We construct the first dataset for code aesthetics, AesCode-3588 and introduce the first bench- mark, OpenDesign, benchmark specifically designed to assess webpage design aesthetics.
We propose a novel reward framework for code aesthetics, agentic reward feedback, and com- bine it with GRPO algorithm for more effective model training in code aesthetics tasks.
RELATED WORKS
Aesthetics of AI-Generated Contents. With the rapid advancement of generative artificial intelli- gence (van der Zant et al. 2013 Sakirin & Kusuma 2023 Jovanovic & Campbell 2022). increasing attention has been directed to the aesthetic taste of Al-generated content (ATGC) (Cao et al. 2025 Wu al. 2023) and the alignment between AI aesthetics and human preferences (Zhang et al. 2024
Liao et al. 2025 Ouyang et al. 2022). Previous works include textual aesthetics (Jiang et al. Dilley 2016). which investigates methods to provides cleaner layout and better coherence
2024
of LLM's output (Jiag et al.] 2024). and image aesthetics (Deng et al.) 2017] Wu et al.) 2024). which focuses on assessing and improving the aesthetic quality of images. However, all these meth- ods rely on evaluating static image(s) and may not capable to assess contents like webpages which need interactions. As the growing maturity of Al agents (Achiam et al. 2023 Hurst et al., 2024), it becomes possible to integrate interactive evaluation into the contents generated by large language models, thereby providing more comprehensive and systematic feedback.
Reward Systems in Reinforcement Learning. In reinforcement learning, the reward serves as a scalar feedback signal that quantitatively evaluates the immediate desirability of an agent's 's actions, thereby guiding the learning process toward behaviors that maximize cumulative long-term return (Kaelbling et al. 1996). In the context of training large language models, the sources of reward can
2

---

Published as a conference paper at ICLR 2026
#2, Static Aesthetics Agent
HTML Data Construction
Screcrshoting Webpage
GFT-3 Zusging
Scncoshht
Finering
Use Iratnaction
Pltt Data Construction
System Prompt
Model Judging
Tramuriio Eetrect Code Ser
Aesicete 2004
cpprr salet parvefy by stoge
vith took uuder delgo tnd latulis duttom tork
Model Traiiin
AccCode
with GAPO
Gone come Aettaiti and ruida bily (3 paiat polnts
jui
User
Cooo pizm mosmee whbote with fox the munu andiceliss enforing
LLM
Agent
Okay. delide politdd IIMEL wehnagg wit undeddes CSS poacctos.
Teed
Screenshot
Execution Agent
Or Outonn
Hosting webpage
Action
GUT Agent
Agent Haviin and Trajectry
T P
Scavio
paan elenstts ineract
Chosss
Andyos
Aaddid
Faled Scrs-
putted SOme
Rusting
boore each rieshaa (.0.1)
Teld sore 1-0-12
Hacty.
Total reward
Wexec
Texec
Wstatic
"static
Winteract
rinteract
Figure 1: Overview of the AesCoder pipeline, which integrates data construction, model training. and a weighted scoring mechanism. GRPO-AR coordinates performing GRPO with three special- ized reward agents-Fxecution Static Aesthetics, and Interactive Aesthetics for comprehensive reward feedback.
be broadly categorized into two main types: (i) Model-based Rewards: This approach utilizes pre- trained reward model generate feedback (Ouyang et al. 2022, Christiano al. 2017] [Wagg al.]
2024a Cui et al. 2023). These models encode human preterences or expert knowledge providing an automated and scalable source of reward. (ii) Rule-based Rewards: This type of reward is generated
directly from human-definee rules or logic (Shao et al.) 2024] Xie et al. 2025 Mu et al., [0224) However, in complex tasks, relying solely on single source of reward can induce biased behaviors, ultimately driving optimization in incorrect direction. Some works have been attempting to agents, which combine human preference rewards with verifiable signals, to provide more reliable rewards (Peng et al. 2025
THE AESCODE-3523 358K DATASET
To investigate code aesthetics, we focus on domains where both the visual outcome and the im- plementation style matter. In this context, two representative areas are considered: Python-based plot generation, which emphasizes clarity and expressiveness in visualization, and webpage design, where aesthetic factors directly influence layout and user experience. In this section, we introduce AesCode-358K, large-scale supervised citruction-tuning dataset designed for two key areas of code aesthetics.
3.1 PYTHON BASED PLOT DATA CONSTRUCTION
We adapted instructions from the existing VisCode-200K dataset (Ni et al. 2025). While the orig- inal dataset contains 200K data points, found that some of the Python code snippets were not
3

---

Published as a conference paper at ICLR 2026
executable. To ensure high quality, used Qwen3-Coder-488B-AA3 Instruccttt Team 2025 Hui et al. 2024) to regenerate the Python code.
We enforced quality control in two ways. First, limited the Python environment to essential libraries like matplotiib, seaborn, and plotly to prevent unexpected imports. Second, we validated the code's executability using Jupyter Notehook runtime checks, ensuring that the gener- ated code runs without errors and produces the correct visualizations. After this rigorous filtering, obtained 158K high-quality plot data points.
3.2 WEBPAGE DESIGN DATA CONSTRUCTION
We developed a four-step process to create a large-scale webpage design dataset. First, we used GPT-4o to generate a seed keyword corpus across five webpage categories: General Website, 3D Design, Data Visualization, Game Dev, and UI Component. Next, GPT-4o was used to produce di- verse webpage design instructions from these keywords. We then projected the instructions into an embedding space and applied t-SNE visualization to examine category overlap. To remove redun- dancy, further applied large-scale clustering and retained only representative samples, resulting in a refined instruction dataset (details in Appendix [..2). Finally, we employed GPT-5 (OpenAl 2025) and Cooen---ddr-- B- instruction. We present dataset statistics and keyword generation prompts in Appendix B
Team 2025 to generate HTML code for each
To ensure the quality of the generated HTML code, we first confirmed that was executable. We then rendered the webpages using playwright [ and selen um and asked GPT-5 to score the two outputs based on their rendered images. We selected the code with the higher score our final data.
4 AGENTIC REWARD FRAMEWORK
For coding tasks, mainstream reward signals typically include execution or unit test success (Gehring et al. 22024 Fu al. 2023). process-awand reward models (Le et al.) [0022] Dai al. 2025), and human preference feedback (Shen et al. 2023). However, these approaches mainly focus on textual modality and lack vision-oriented reward signals, rendering them unsuitable for evaluating code aesthetics. In visually grounded code generation, we highlight three essential dimensions:
Code Executability. The generated code must successfully, which forms the fundamental requirement of all code-related tasks.
Static Aesthetics. This dimension captures the visual quality of the rendered output. An ef- fective design should be concise, stul-stru and visually coherent, with elements properly aligned and exhibiting clear sense of design.
Interactive Aesthetics. Beyond static visuals, interactive aspects are crucial for web- especialyy those featuring 3D objects or browser-based games. This dimension eval- uates whether page elements are not only visually appealing but also functionally meaningful and reasonably interactive.
Based on these dimensions, propose an agentic reward framework that leverages a multi-agent system to assess each aspect, integrates their evaluations, and generates comprehensive feedback for webpage design from multiple perspectives.
4.1 EXECUTION AGENT
The execution agent verifies whether the model's output is executable and reports the result to the feedback system. Specifically, it assigns 8exe 1 if the output passes all validations, and 8exec -1 otherwise. For a raw model output, the agent first attempts to extract the HTML code from the html block; if not found, the entire output is treated as HTML. Given that web browsers tolerate many structural and syntactic errors, strict execution checking is unsuitable for HTMLL Instead, we use
httos://plawwricht... https: www. dev/
4

---

Published as a conference paper at ICLR 2026
P to implement rule-based HTML checker to validate the basic syntax. The detailed rules can be seen in Appendix G.7
HTMLHint
4.2 STATIC AESTHETICS AGENT
The static aesthetics agent evaluates visual quality using full-page webpage screenshots. For an HTML file, first hosts the page locally using playwright in headless mode, then captures a full-page screenshot for subsequent visual assessment. We identify three dimensions essential for evaluating a webpage screenshot:
Instructional Alignment. Evaluates consistency between the page's style and user instructions.
Visual Elements. Assesses the effective use of modern design features such as lighting, trans- parency, and gradients.
Layout and Cohesion. Examines whether the structure is functional, responsive, and visually coherent, with concise yet design-aware typography.
We select GPT-5 (OpenAl 2025) the judge for its strong multimodal reasoning ability. Given an HTML file generated by the model under a prompt, the page is first rendered into a static image. Using chain-of-thought approach (Wei et al. 2023), the judge evaluates this image and provides both a score and a rationale for each dimension. While both scores and explanations are required to ensure reliable evaluation (Wei et al. 2023 Yao et al. 2023), retain only the final aggregated score as the output of the static aesthetics agent. The detailed prompts are provided in Appendix G.2
4.3 INTERACTIVE AESTHETICS AGENT
For webpage design, evaluation based only static screenshots is insufficient, as it overempha- sizes visual appearance while neglecting usability. This issue particularly critical for interactive webpages such 3D design platforms browser-based games. To address this, we introduce the in- teractive aesthetics agent, which autonomously navigates, explores, and interacts with webpages to provide usability-awan feedback. Given the HTML code, the agent launches the page in a headless environment, interacts with its elements, and evaluates their functionality. We adopt Web Voyager the base framework and GPT-4o (OpenAl 2024) as the multimodal model.
(He al. 2024
Agent Planning. At the start of evaluation, the agent generates an initial list of interaction can- didates by reasoning about which elements are most relevant to the user instruction and webpage content. It then ranks these candidates and selects the top N for execution. To ensure evaluations re- main offline, interactions requiring internet access (e.g., social media logins) are excluded, focusing only the core webpage functionality.
Agent Interacting and Scoring. The agent then executes the planned interactions step by step, recording whether each attempt succeeds fails. After completing all interactions, it outputs binary score list indicating success (1) or failure (0) for each action, and aggregates them into a final =i=1 85. This score is then returned to the agentic reward framework (see Appendix G.3 for the full prompt).
interaction score: Sinteract
Discussions. Current web agents can handle most webpage operations (He et al. 2024). but may still struggle with certain corner cases, such confusing webpage elements or being misled by irrelevant textual content (Cemri et al. 2025) Wang et al. 2024b). Such agent failures lead score of 0 in the corresponding iteration, since we assign a score of only when the webpage responds correctly. This may cause the agent to make incorrect judgments, resulting in scores lower than the true values. On the other hand, agent failures also partially reveal non- standadd or sub-optimal aspects of webpage design. Therefore, despite these limitations, using web agents evaluators provides a reasonable proxy for assessing overall webpage aesthetics and interactivity.
https::/htmlhint..cm

---

Published as a conference paper at ICLR 2026
4.4 REWARD AGGREGATION
The results from the three agents are integrated by the agentic reward framework, which jointly evaluates execution, static aesthetics, and interactive aestheties to provide comprehensive feedback on each webpage. Let Texse static and Finteract denote the rewards from the respective agents. The overall reward is then computed as
Wexee Texee
Watatio
Tstatic + Winteract Tinteracts
where w represents the weight assigned each agent.
AESCODER TRAINING
5.1 STAGE I: SUPERVISED FINE-TUNING ON AESCODE-358K
We perform supervised fine-tuning on two different model with different parameter scales on our AesCode-3588 dataset: Qwven3-4B-Instmet-2500 (Team) 2025) and Sween.5-Coddr--7--nnstt (Hui et al.) 2024). This validates the generalizability of AesCode-358K dataset and establish a robust foundation for next stage reinforcement learning.
5.2 STAGE II: REINFORCEMENT LEARNING WITH AGENTIC REWARD FREDBACK
After supervised fine-tuning in stage I. the model acquires substantial high-quality knowledge. How- ever, the model at this stage still exhibits limited generalization beyond the training distribution (Chu et al. 2025), especially in webpage design tasks. This limitation highlights the necessity of rein- forcement learning (RL), which allows the model to adapt more flexibly and robustly to diverse and unseen scenarios. Thus, we perform reinforcement learning using the GRPO-AR method, which integrates the GRPO (Shao et al. 2024) algorithm with our Agentic Reward framework to enhance the model's ability.
Data Preparation for RL. For avoiding overlap with the data in AcsCode-358K, which the model has already "seen" in stage we pick 20K RL data from WebSight v0.2 dataset (Laurençon al., 2024). However, the user instructions WebSight v0.2 are not categorized, so we use the original user instructions seeds and use GPT-4o (OpenAl 2024) rewrite the instructions for clearer semantic expression. Prompts refer to Appendix G.6
GRPO with Agentic Reward. To generalize model's webpage design ability, we adopt our agen- tic reward system as a reliable and robust reward provider and perform reinforcement learning using GRPO (Shao al. 2024) algorithm. We call this training method as GRPO-AR. For each prompt p in our RL dataset DRL. GRPO-AR samples a group of outputs {01, 02,.... ... ... og) from the old policy model Te.. and our agentic reward framework will give each output total reward r, from execution, static aesthetics, and interactive aesthetics perspectives respectively, yielding G rewards (11, respectively. The advantage Åi, can be caculated follows:
mean(r)
A.
std(r)
Accordingly, the policy model is optimized by maximizing the GRPO objective under our agentic reward framework (GRPO-AR):
Janso (0) Elp ~ DEL (o. ~
nown (Op)]
To o..)) A.s. clip oree 1-,, 1.) ( DDa -DDn (sollaad) (3)
min
Tan local, os,ce) (oop(o)
THE OPENDESIGN BENCHMARK
Design Arena" is a widely used platform for benchmarking web page design, supported by a com- munity of hundreds of thousands of voters. allows users to submit web page designs and receive
https://dest gnarena...... .ii/

---

Published as a conference paper at ICLR 2026
community feedback through voting. While effective, this voting process is time-consuming and impractical for large-scale evaluation.
To address this limitation, introduce the OpenDesign Benchmark, which enables efficient and automated assessment of web page designs using large language models. The benchmark evaluates both static and interactive aspects of design and includes 840 real-world web page cases. A detailed breakdown of categories and their case counts is provided in the Appendix C
We assess model performance from two perspectives: static aesthetics and interactive aesthetics. Static evaluation: given a prompt, the HTML generated by a model is rendered into a static image. The prompt and the image are then assessed by the static aesthetics agent (see Sec. 4.2) which pro- duces a static aesthetics score. In- teractive evaluation: using the same prompt and HTML code, the interac- tive aesthetics agent (see Sec.(4.3) as- signs an interactive aesthetics score. The final benchmark score for a model is obtained by averaging these results across all benchmark cases.
0.00
0.775
9:79
2--043 3u7 8.50 065
8:70 0.7
To evaluate the quality and reli- ability of the OpenDesign bench- mark, we adopt two complemen- tary perspectives: (1) ranking consis- tency between OpenDesign and De- sign Arena, and (2) alignment be- tween LLM scoring and human preference.
Enckatar
(b) Agreement rates among model rankings between Open- GPT and human evaluators. H-i
(a) Comparison of relative
Design and Design Arena.
refers to human evaluators.
To assess the reliability of our OpenDesign Benchmark, we compare the rankings of 10 mainstream foundation models against the Design Arena leaderboard We measure consistency using Spear- man's and Kendall's rank correlation coefficients, obtaining strong agreement: Spearman = 0.98
(p < 1.5 10-9) and Kendall 0.91 (p < 3.0 10-5). Additionally, OpenDesign achieves 66.7% top-3 and 80.0% top-5 overlap with Design Arena. These results indicate that OpenDesign closely reflects large-scale human judgment. Figure plots model ranks across both benchmarks. Points align closely with the diagonal, confirming OpenDesign reliable proxy for human preferences in webpage aesthetics.
We sampled 200 HTML page pairs generated by the 10 models under the same prompts. Two eval- uator groups-GPT judge and 10 humans (3 professors, graduate comparisons (win/tie/lose)) yielding 2,000 annotations. Figure 2b shows agreement ratios: human- human = 68.7%, GPT-human = 80.9 80.9%. These are comparable to MT-Bench results (66% and 70%, respectively) (Bai al. 2024 Zheng et al.) 2023), supporting LLM-as-at -uddue effective, bust method for assessing code aesthetics.
pairwise
7 EXPERIMENTS AND RESULTS
7.1 EXPERIMENTAL SETUP
We evaluate the model's plot generation using PandasPlot (Galimzyanov et al. 2025) with the head descriptor and vis mode. For each case, the model generates code from an instruction; executability is checked, and if image is produced, is compared the ground truth. GPT-40 scores each case from to 100. This results in three quantitative results, (i) error rate, which refers to the portion of cases do not pass the executability check, (ii) average score, which is the average GPT-4o score among all test cases, and (iii) good rate, which refers to the portion of scores higher than 75. Webpage design ability is assessed using our OpenDesign benchmark (see Section (6) Training settings are provided in the Appendix E
Rankings are taken as of Septemher 22, 2025; Design Arena updates dynamically.
7

---

Published as a conference paper at ICLR 2026
Table 1: Performance comparison between proprictary and open-source models across various benchmarks. In PandasPlotBenchh, Err., Avg., Good. refer to error rate, average score, good rate respectively. In OpenDesign, Align., Aes., Struct. refer to the three score perspectives: instructional alignment with user instruction, visual elements aesthetics, and structural cohesion respectively. To- tal. means the total score of sum of three aspects' scores, and InterAes. refers to the score of interactive evaluation stage. Note: Lower is better for Err., higher is better for all other metrics. Best results are bold, second-best results are underlined (among all open- open-source models together).
PandisslotRRnch
OpenDesign
Model
Size
Avg. (1) Good.
Static Aestheties
InterAes, (1)
Align. (1)
Acs. (*)
Struct. (*)
Total. (?)
Praprictary Models
GPT-4o GPPT-4o-mini
0.15
0.57
14.29
14.13
12.77
41.19
0.40
GPT-4o
0.09
16.90
16.05
15.13
48.08
0.44
GPT-4.1
0.09
69
0.61
23,53
21.99
20.27
65.79
0.74
GPT-5 (minimal)
0.04
30.38
25.94
24.71
81.03
1.37
Claude Sonnet
0.04
74
0.66
29.60
25.92
25.53
81.05
0.92
Open-Source Source Large Language Models
Owen3-Coders 303
30B
0.07
0.62
27.04
23.79
22.75
73.66
0.52
GLM-1-328-0414
32B
0.07
70
0.59
24.67
22.90
21.80
69:40
0.48
GL---45--ir
1108
0.08
71
0.63
29.29
24.83
24.04
78.16
0.93
Swen--Coder480B-AAS
4808
0.05
73
0.66
30.13
25.16
24.62
79.90
0.70
DeepSeek-V3.1
685B
0.09
29.35
24.37
24.00
77.72
0.88
DeepSeek 0528
685B
0.08
0.63
30.02
24.69
24.09
78.86
0.77
Open Source Language
Owen3-4B 1a0-----t-ttttt-250000
4B
0.13
65
0.55
27.52
23.01
22,73
73.26
0.67
Qwen2.5 Coder- 7a-ee-7B-Insseee
7B
AcsCoder-4B (Ours)
4B
0.09
70
0.63
30.42
26.19
25.31
81.92
1.04
AesCoder:77 (Ours)
30.03
25.98
25.18
81.23
7.2
MAIN RESULTS
As shown in Table 1 both AesCoder-4B and AesCoder-7B achieve consistent improvements their respective baselines. On PandasPlotBench, they achieve lower error rates and higher reliabil- ity, indicating stronger capability in generating correct plotting code. On OpenDesign, AesCoder achieves substantial improvements in both static aesthetics (alignment, visual appeal, and struc- ture) and interactive aesthetics, surpassing all other open-source models. In particular, AesCoder matches or outperforms models with 30B-685B parameters, establishing new state-of- tte-thh-rrtttt results among open-source systems.
When compared with proprietary models, AesCoder-4B not only surpasses GPT-4o and GPT-4.1 on both PandasPlotBench and OpenDesign, but also delivers results competitive with substantially larger systems. Although GPT-5 and Claude Sonnet 4 still retain slight overall advantage, our models achieve comparable scores across several aesthetic dimensions. These findings underscore the effectiveness of GRPO-AR, demonstrating that reinforcement learning with agentic reward feed- back consistently enhances performance across different architectures and scales.
We further conducted human evaluation (Appendix F). and the results show that AesCoder-4B consistently outperforms strong open-source baselines (GLM-4-32B-04)) and Qwen3-Coder-30B- A3B- Ans-tuut, which further validates results.
7.3 ANALYSIS
Generalization of agentic reward. We further analyze the reward dynamics during reinforcement learning, as lustrated in Figure 3 Both D0wen2.5-Coder-7B-- exhibit steadily
0.55
Qwen3:
SFT and Dowen3-4B-Instrut--25 increasing reward scores with training steps. This consis- tent upward trend indicates that the agentic reward frame- work provides stable and informative feedback, enabling continuous improvement across different model families and sizes. The results highlight the robustness of the
0.45
0.40
0.35
Training
Figure 3: Reward curves during GRPO- AR.
8

---

Published as a conference paper at ICLR 2026
framework as general training signal, independent of specific architecture choices.
Effeet of Agentic Reward. To isolate the contribution
of the proposed agentic reward, we conduct controlled comparison against a variant that does not incorporate Specifically, instead of leveraging the full agentic reward framework, we directly employ the underlying reward model to score model-generated HTML outputs along three static Aligmment, Visual Design and Aesthetics, and Structural Coherence and Usa use these scores as the sole reward signal (see Appendix G.44 for the exact prompt). The policy optimization strictly follows the same procedure as in 55.2 with the updates computed according to Eq. 3 thereby ensuring fair comparison.
As reported in Table 2 this simplified variant consistently underperforms the full method that integrates agentic reward feedback. The perfor- mance gap highlights that merely reusing the reward model in a static fashion is insufficient. agentic reward framework,
Table 2: Comparison with DPO, RFT, and abla- tions on Agentic Reward for Qwen3-4B -wen3-4B-Instruut- 2507 and Qwen2.5 Co-de--od---nsteeee
Training Strateyy
Align
Struet
LaterAus
In contrast, which incorporates multi-perspective evalua- tions including execution, static, and interactive aesthetics, provides richer and more reliable feedback. These results demonstrate that agen- tic reward is essential for aligning the model with both functional correctness and human- perceived aesthetics.
Raseline DPO
25:27
24.
0.62
24.67
28.79
25.31
24:38
0.70
GRIO-AR AR wio Ageatic Reward (ablation)
29.15
25.20
24.67
0.71
GRPO-AR w/ Agentic Reward (ours)
30.42
26.19
25.31
1.04
Baseline
28,85
25.23
24.37
0.70
RFT
20.75
25.35
24.85
0.75
DIO
29.75
25.33
24.87
0.71
GRPO-AR Ageanic Reward (ablation)
24.41
GRPO-AR Agentic (ours)
25.18
Comparison with DPO and RFT. To further validate the effectiveness of our proposed method GRPO-AR, we additionally compare it with two RLHF methods: Direct Preference Optimization (DPO) (Rafailov et al., (204)) and Rejection Sampling Fine-Tuning (RFT) (Yuan et al. (023). Both methods are applied to the Stage checkpoint TTr using the same training data as Stage II to ensure fair comparison. Implementation details of DPO and RFT are provided in Appendix [D As shown in Table our method consistently surpasses both DPO and RFT on OpenDesign across static and interactive aesthetics. These improvements highlight that incorporating agentic reward feedback not only enhances the visual quality of generated webpages but also strengthens their usability and structural robustness, confirming the superiority of GRPO-AR.
8 CASE STUDY
We further conduct case studies on the OpenDesign benchmark to qualitatively compare AesCoder- 4B with Claude Sonnet (Anthropic) 2025) and DeepSeek-RR R1-0k22 (DeepSeek-Al et al. 2025). We select five representative cases from the five categories in OpenDesign for comparison. As illustrated in Figure AcsCoder-4B achieves results that are superior to or on par with ofate-of-the-au models across all five web design task categories. These results highlight the effectiveness of our approach in aligning code generation with both usability and aesthetic quality.
CONCLUSION
In this work, we introduce the concept of code aesthetics and present AesCode-358K, OpenDesign, and agentic reward framework (GRPO-AR) that jointly enhance executability, static design, and interactivity in code generation. Through supervised tuning and reinforcement learning with GRPO-AR, our AesCoder models achieve state-of-the-art results on PandasPlotBench and Open- Design, rivaling much larger models. These results demonstrate that multi-agent reward feedback can effectively align coding LLMs with both functional correctness and numan-perceinn aesthetics, paving the way for more capable and user- ser--frienlly coding assistants.
9

---

Published as a conference paper at ICLR 2026
AesCoder-4B (Ours)
Claude Sonnet
SrepSeek-R1-Ose2
Premium Products Your Furry Friend
Premium products Shar keeep
Furry Friend
Prompt: Create wur-frionnll webitte landing pagee dedicated welling dog-relaved products, encuring wany novigation appeallnn desiga for pet owners.
Tarto
Prompt: a template display trffic wing graphs.
Prompt: Create a blocky virnal landscape remintscent of Minecruft, where players can explore and interaat with a pivelated 3D world.
Prompt: on interactive Toe-To-- game for aliowing two players marking on grid.
Prompt: Create user interface for partners aiming improve secual compenbtity; explore interests, and understand each other's moods. This helps address like lock of intimary and induces awkwaddeus rejection intricting interactions.
Figure 4: Case study comparing AesCoder-4B and baseline models on OpenDe The categories from top bottom General Website, Data Visualization, 3D Design, Game Dev, UI Component.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, 1lge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical
10

---

Published as a conference paper at ICLR 2026
report. arXiv preprint atXtv:2303.0877,,222
Anthropic. Introducing claude 4. 2025.
May
Anthropic. Claude code: Best practices for agentic coding.
htttps://ww..anthrrr com/
( bodettt-----ddeeeeb----e-------- cheat-practicee 2025. Accessed: 2025-09-25.
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. Program synthesis with large language models, 2021. URL https: waw.iiiiiivv...i. www.:xiww...oo..oooooo.oo htagg//bbss 2108. 07732
Ge Bai, Jie Liu, Xingyuan Bu, Yancheng He, Jiaheng Liu, Zhanhui Zhou, Zhuoran Lin, Wenbo Su, Ticzheng Ge, Bo Zheng, and Wanli Ouyang. Mt-bench-1011 101: A fine-grained benchmark for eval- uating large language models in multi-turn dialogues. In Proceedings of the 62nd Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 7421-7454. Association for Computational Linguistics, 2024. doi: 10. 1tt.5/////222..a////s
URL
http:// doi. ottg www.111666/1/00 http://2222..c....01. 401
Yihan Cao, Siyu Li, Yixin Liu, Zhiling Yan, Yutong Dai, Philip Yu, and Lichao Sun. A survey of ai-generated content (aige). ACM Computing Surveys, 57(5): 2025.
Mert Cemri, Melissa Z. Pan, Shuyi Yang, Lakshya A. Agrawal, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya Parameswaran, Dan Klein, Kannan Ramchandran, Matei Zaharia, Joseph E. Gonzalez, and Ion Stoica. Why do multi-agent Ilm systems fail?, 2025. URL BRRpbttss://axiv
erg/abs. 2503. 13657
Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. Advances in neural information processing sys- tems, 30, 2017.
Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shenghang Tong, Saining Xie, Dale Schuurmans, Quoe V. Le, Sergey Levine, and Yi Ma. Sft memorizes, rl generalizes: A comparative study of foundation model post-training, 2025.
17161
Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Wei Zhu, Yuan Ni, Guotong Xie, Zhiyuan Liu, and Maosong Sun. Ultrafeedback: Boosting language models with high-quality feedback. 2023.
Ning Dai, Zheng Wu, Renjie Zheng, Ziyun Wei, Wenlei Shi, Xing Jin, Guanlin Liu, Chen Dun, Liang Huang, and Lin Yan. Process supervision-guideed policy optimization for code generation, 2025. URL https:::: // / arxiv. www.w.ogg.aaaaaa/..aoo//aaaa.aaa 2210. 17621
DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, F. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu l.,, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang. Bochao Wu, Bei Feng. Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu. Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang. Lecong Zhang. Liang Zhao, Litong Wang. Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang. Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang. Qinyu Chen, Qiushi Du, Ruiqi Gc, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. Chen, R. I. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun, Tian Pei. Tianyu Sun, T. Wang, Wangding Zeng, Wanjia Zhao, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang, W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li. Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xi- aosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang. Xinxia Shan, Y. K. Li, Y. Q. Wang, Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng
11

---

Published as a conference paper at ICLR 2026
Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang, Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu. Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. Deepseek-rl: Incentivizing reasoning capability in llms via reinforce- ment learning. 2025. hntts://:/aae
org/ /tlos/2501..00 12948
Yubin Deng, Chen Change Loy, and Xiaoou Tang. Image aesthetic assessment: An experimental survey. IEEE Signal Processing Magazine, (4(4):80-106, 2017.
Paul C Dilley. Textual aesthetics. The Red Monastery Church: Beauty and Asceticism in Upper Egypt, pp. 175, 2016.
Qiang Fu, Xiao Han, Wei Yang, Deheng Ye, Kaiwen Xiao, Jiate Liu, and Yigin Zhu. Ritf: Rein- forcement learning from unit test feedback, 2023. htttss///aax wwwp:///.rrgbbb/2333....
04349
Timur Galimzyanov, Sergey Titov, Yaroslav Golubev, and Egor Bogomolow. Drawing pandas: A benchmark for lims in generating plotting code, 2025. URL https://arxiv..rr// /bbs/
2412. 02764
Jonas Gehring, Kunhao Zheng, Jade Copet, Vegard Mella, Taco Cohen, and Gabriele Syn- naeve. Rlef: Grounding code iims in execution feedhack with reinforcement learning. ArXiv, abs/2410.02089, 2024.
Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, Hao Yu, Hongning Wang, Jiadai Sun, Jiajie Zhang, Jiale Cheng, Jiayi Gui, Jie Tang, Jing Zhang. Juanzi Li, Lei Zhao, Lindong Wu, Lucen Zhong, Mingdao Liu, Minlie Huang, Peng Zhang. Qinkai Zheng, Rui Lu, Shuaiqi Duan, Shudan Zhang, Shulin Cao, Shuxun Yang. Weng Lam Tam, Wenyi Zhao, Xiao Liu, Xiao Xia, Xiaohan Zhang, Xiaotao Gu, Xin Lv, Xinghan Liu, Xinyi Liu, Xinyue Yang, Xixuan Song, Xunkai Zhang, Yifan An, Yifan Xu, Yilin Niu, Yuantao Yang, Yueyan Li, Yushi Bai, Yuxiao Dong, Zehan Qi, Zhaoyu Wang. Zhen Yang, Zhengxiao Du Zhenyu Hou, and Zihan Wang. Chatglm: A family of large language models from glm- 130b to glm-4 all tools, 2024.
Jiawei Gu, Xuhui Jiang, Zhichao Shi. Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Saizhuo Wang. Kun Zhang. Yuanzhuo Wang. Wen Gao, Lionel Ni, and Jian Guo. A survey on Ilm-as-a-judge, 2025. URL http::s//rxxvv.orgo
2241. 15594]
Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y. Wu, Y. K. Li, Fuli Luo, Yingfei Xiong, and Wenfeng Liang. Deepseek-coder; When the large language model meets programming the rise of code intelligence, 2024. URL https:
//arxiv.org// abs/ /201. 14196
Hongliang He, Wenlin Yao, Kaixin Ma, Wenhao Yu, Yong Dai, Hongming Zhang, Zhenzhong Lan, and Dong Yu. Webvoyager: Building an end-to-end web agent with large multimodal models, 2024. /RRl/tttps://axxi.oorrg
13919
Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Kai Dang, et al. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409. 12186, 2024.
J. D. Hunter. Matplothib: 2d graphies environment. Computing in Science & Engineering, 9(3): 90-95, 2007. doi: 10.1 10.1009/CSS.220..555
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21226, 2024.
12

---

Published as a conference paper at ICLR 2026
Plotly Technologies Inc. Collaborative data science, 2015. URL htttttt//ppot....
Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang. Armando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code, 2024. URL https: //axiiv///xivvva//aa org/ abs/
07974
Lingjie Jiang, Shaohan Huang, Xun Wu, and Furu Wei. Textual aesthetics in large language models. arXiv preprint arXiv:2411.02950 2024.
Mladan Jovanovic and Mark Campbell. Generative artificial intelligence: Trends and prospects. Computer, 55(10):107-12,, 2022.
Leslie Pack Kaelbling, Michael L Littman, and Andrew W Moore. Reinforcement learning: A survey. Journal of artificial intelligence research, 4:237-285, 1996.
Hugo Laurençon, Léo Tronchon, and Victor Sanh. Unlocking the conversion of web screenshots into html code with the websight dataset, 2024.
Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, and Steven Hoi. CodeRL: Mastering code generation through pretrained models and deep reinforcement learning. In Al- ice H. Oh, Alckh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neu- ral Information Processing Systems, 2022. URL /tttp:://penreeveew net forttfformmiss WaGvb70zySA
Zhichao Liao, Xiaokun Liu, Wenyu Qin, Qingyu Li, Qiulin Wang, Pengfei Wan, Di Zhang, Long Zeng, and Pingfa Feng. Humanaesexpert: Advancing a multi-modality foundation model for human image aesthetic assessment. arXiv preprint arXiv:2503.23997, 2025.
Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation, 2023.
URL https:
awt./asssss... 01210
Tong Mu, Alec Helyar, Johannes Heidecke, Joshua Achiam, Andrea Vallone, Ian Kivlichan, Molly Lin, Alex Beutel, John Schulman, and Lilian Weng. Rule based rewards for language model safety. Advances Neural Information Processing Systems, 37: 178877777100800 2024.
Dang Nguyen, Jian Chen, Yu Wang, Gang Wu, Namyong Park, Zhengmian Hu, Hanjia Lyu, Junda Wu, Ryan Aponte, Yu Xia, Xintong Li, Jing Shi, Hongjie Chen, Viet Dac Lai, Zhouhang Xie, Sungchul Kim, Ruiyi Zhang, Tong Yu, Mchrab Tanjim. Nesreen K. Ahmed, Puneet Mathur, Seunghyun Yoon, Lina Yao. Branislav Kveton, Thien Huu Nguyen, Trung Bui, Tianyi Zhou, Ryan A. Rossi, and Franck Dernoncourt. Gui agents: A survey, 2025. URL (tRltttts:///rxtt
org atg//bs//a 2412. 13501
Yuansheng Ni, Ping Nie, Kai Zou, Xiang Yue, and Wenhu Chen. Viscoder: Fine-tuning Ilms for executable python visualization code generation. arXiv preprint arXiv:2500.033330, 2025.
OpenAl. Hello gpt-4o. httpss///ggeea...co//////eeeeeee
2024. Accessed:
2025-09-13.
OpenAl. Introducing gpt-5. 2025. Accessed: 2025-09-13.
httea://ogeasicccommmmeee///tttt gpt 5/
OpenAl. Gpt-5 system http://ss...eeaa....onnn///////s gust 2025.
Au-
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to fol- low instructions with human feedback. Advances in neural information processing systems, 35: 27730-2777 277444-7744, 2022.
Hao Peng, Yunjia Qi, Xiaozhi Wang, Zijun Yao, Bin Xu, Lei Hou, and Juanzi Li. Agentic reward modeling: Integrating human preferences with verifiable correctness signals for reliable reward systems. arXiv preprint arXiv:2502. 19328, 2025.
13

---

Published as a conference paper at ICLR 2026
Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. Direct preference optimization: Your language model secretly a reward model, 2024.
abs /000 18290
Tam Sakirin and Siddartha Kusuma. A survey of generative artificial intelligence techniques. Baby- lonian Journal of Artificial Intelligence, 2023:10-14, 2023.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models. arXiv preprint arXiv:2402.033300 2024.
Bo Shen, Jiaxin Zhang, Taihong Chen, Daoguang Zan, Bing Geng, An Fu, Muhan Zeng, Ailun Yu, Jichuan Ji, Jingyang Zhao, Yuenan Guo, and Qianxiang Wang. Pangu-coder2: Boosting large language models for code with ranking feedback, 2023. URL https:://axxiiv org. abs/ 2307. 14936
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rihí' framework. arXiv preprint arXiv: 2409. 19256, 2024.
Qwen Team. Qwen3 technical report, 2025. URL
Tijn van der Zant, Matthijs Kouw, and Lambert Schomaker. Generative artificial intelligence. In Philosophy and theory of artificial intelligence, pp. 107-120. Springer, 2013.
Haoxiang Wang, Wei Xiong, Tengyang Xie, Han Zhao, and Tong Zhang. Interpretable preferences via multi-objective reward modeling and mixture-of-experts. arXiv preprint arXiv:2406. 12845, 2024a.
Lei Wang. Chen Ma. Xueyang Feng, Zeyu Zhang, Hao Yang. Jingsen Zhang, Zhiyuan Chen, Ji- akai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Jirong Wen. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6), March 2024b. ISSN 2095-2236. doi: 10. 10.1007/s1 1707/0021774--44032311 URL http: doi org/ 10.
Michael L. Waskom. seaborn: statistical data visualization. Journal of Open Source Software, (60):3021, 2021. doi:
URL
03C21
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models, 2023. URRl:tttps//aaxx..rrrr abs. 2220
Jiayang Wu, Wensheng Gan, Zefeng Chen, Shicheng Wan, and Hong Lin. Ai-generated content (aige): A survey. arXiv preprint 06X6v:2304.06633, 2023.
Xun Wu, Shaohan Huang, and Furu Wei. Multimodal large language model is human-aligned annotator for text-to-image generation. arXiv preprinl arXiv:2404. 15100, 2024.
Tian Xie, Zitian Gao, Qingnan Ren, Haoming Luo, Yuqian Hong. Bryan Dai, Joey Zhou, Kai Qiu, Zhirong Wu, and Chong Luo. Logic-rl: Unleashing 11m reasoning with rule-based reinforcement learning. arXiv preprint arXiv: 14768, 2025.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models, 2023. URL httpt://ps://xiv erg/aba/2e 2210 03629
Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Keming Lu, Chuanqi Tan, Chang Zhou, and Jingren Zhou Scaling relationship on learning mathematical reasoning with large language models, 2023. URL peetts://arrivve org/ abs 2308 01825
Chaoyun Zhang, Shilin He, Jiaxu Qian, Bowen Li, Liqun Li, Si Qin, Yu Kang, Minghua Ma, Guyue Liu, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, and Qi Zhang. Large language model- brained gui agents: A survey, 2025. URL
wwwp:///gg////aa.....o 18279
14

---

Published as a conference paper at ICLR 2026
Miaosen Zhang, Yixuan Wei, Zhen Xing, Yifei Ma, Zuxuan Wu, Ji Li, Zheng Zhang, Qi Dai, Chong Luo, Xin Geng, et al. Aligning vision models with human aesthetics in retrieval: Benchmarks and algorithms. Advances in Neural Information Processing Systems, 37:863999 2024.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging Aim-as-ajuugee with mt-bench and chatbot arena, 2023. URL BRRLittts://axiv..o
abs/ 2000/2306.om 05685
Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yonggiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models, 2024.
URL paattss:/aazttae
15

---

Published as a conference paper at ICLR 2026
A LLM USAGE STATEMENT
A large language model (C used aid and polish the writing of the paper, including minor grammar correction and language refinement.
B DETAILS OF WEB PAGE DATA CONSTRUCTION
B.1 KEYWORD CORPUS AND INSTRUCTION GENERATION
We classified webpages into five categories: General Website, 3D Design, Data Visualization, Game Dev, and UI Component. Using GPT-4o, generated 9K seed keywords for the General Website category, and 2.5K keywords for each of the remaining four categories. Table 3 summarizes the distribution.
Table Seed keywords statistics across categories.
Category
General Website
3D Design
Data Visualization
Game Dev
Component
Seed Keywords
9,000
2,500
2,500
2,500
2,500
Based the seed corpus, GPT-4o asked to generate 20 non-redundant and semantically di- verse instructions for each keyword. This resulted in a total of 400,000 webpage design instructions for further processing.
B.2 SEMANTIC ANALYSIS AND DEDUPLICATION
(3072 dimensions). From each category, 2,000 instructions were randomly sampled and visualized with t-SNE (perplex- ity 30, max iter 1000). As shown in Figure [5] the dataset exhibited significant overlaps across categories, along with several dense clusters.
We embedded all instructions using opena wwweaii-tee---ebbbbdin--------
To filter redundancy, applied K-Means clustering with K 200K the embedded vectors and kept only the sample nearest to each cluster center. This resulted in a refined dataset of 200K instructions. The t-SNE visualization of the refined dataset shows clearer class boundaries and reduced overlap across categories, demonstrating the effectiveness of our filtering.
RC3
Suy
(a) t-SNE of raw data
(b) L-SNE of filtered data
Figure 5: Visualization of instruction embeddings before and after filtering.
16

---

Published as a conference paper at ICLR 2026
Table 4: Distribution of OpenDesign Benchmark Categories (Total: 840 cases)
General Website
3D Design
Data Visualization
Game Dev
Ul Component
Total
60.9%
14.6%
4.8%
13.6%
4.9%
100%
C OPENDESIGN BENCHMARK CATEGORIES
D IMPLEMENTATION DETAILS FOR DPO AND RFT
In this section, describe the construction pipeline of training data for both DPO and RFT used 7.3 We adopt the same set of queries as GRPO-AR for offline sampling. For each query 4, sample N responses from the SFT policy TTrT. yielding
( = o I
(4)
reward model Rs then scores each response conditioned on q:
R(q) r(o, g) o, E O
}. where r(o (()1g) (
(5)
DPO. For DPO, construct preference dataset by taking, for each q, the highest- and lowest- scoring responses:
DDPO = (g, Ow or) Ow arg max r(o q), or = arg min r(o ()) oEO(I)
(6)
oEO(O)
We then optimize Te (initialized from TeseT) with the standard DPO objective (Rafailov et al.
2024
max (ao ~ Doro [og a (1og
(a(awlg) log mo(ar (g) reser (oilg) ))
weger (owlg)
where (-) the sigmoid and
a scaling hyperparameter.
RFT. For RFT, we select only the top-scoring response per query:
DRET = (g.0) arg max r(o 4) oEO(g)
(8)
The model is then trained with a standard supervised objective:
o
(R) -ig.o) DRFT
log ne (ar 1gl 010-1)
(9)
Implementation. We implement both DPO and RFT with LLaMA-Factor, (Zhngg et 2024 For fair comparison with GRPO-AR, we keep the same learning rate, batch size, and the total number of training samples as in Stage II.
TRAINING SETTINGS.
For stage I, all models are trained for epochs with the AdamW optimizer, employing a 10% linear warmup followed by cosine learning rate decay schedule. The maximum learning rate is set to le-5, with a batch size of 128 and a maximum sequence length of tokens. Training model in the SFT phase takes approximately days on nodes of 8xMI300 GPUs.
For stage II, we use VeRI. (Sheng et al.) 2024) to conduct experiments. By default, we use constant 3 x 10-6 learning rate together with AdamW optimizer for policy model. and use batch size of 64 and micro batchsize of The rollout stage collects 64 prompts and samples responses for each prompt. We KL coefficient 0.001 and 0.5 in Eq. in all experiments. The RL phase
https: whw.tttt...... com hwwpttth...cooooo/oo///a//aa/a/e Wactory
17

---

Published as a conference paper at ICLR 2026
takes approximately days on nodes of 8xM1300 GPUs. In agentic reward framework,
Wexee 0.1, Wstatic 0.8, and Winteract 0.1. Given the currently low success rate of GUI agents (Zhang et al. 2025 Nguyen et al., 2025 He al. 2024), limit the number of interactive elements to 3 during training. Additionalyy ly. when the GUI agent lists the interactive elements, we instruct it prioritize them based their importance. This ensures that the most critical and prominent elements are interacted with, thereby mitigating the impact of the GUI agent's 's limited success rate on our GRPO-AR training.
F
HUMAN EVALUATION
Lose
Win
AcsiCoder 48
AesCoder 78 vis
AesCoder GPE5
AecCnder
GLM 3.444-322 328 328-044
0414
GLM-4-328-0414
AesCoder Owend- Coder- All--isttrct
AesCrder Qwen Coder-30B-A 10----00------teeeee -nttttistttctt
0%
25%
50%
75% 100%
0%
25%
50%
75%
302%
(a) AesCoder 4B
(b) AesCoder 7B
Table 5: Human preference result visualization AesCoder and other models.
To validate the effectiveness of our model, we select four mainstream models, Claude Sonnet 4 An- thropic 2025), GPT-5 (OpenAl 2025), GLMM-433B-0444405 04144(LM al.] 2024) and Qwen3-Coder- 30B-A3B-BI B-Intrrutt (Team 2025) and randomly sampled 100 test cases from OpenDesign, resulting 100 HTML pairs (Taurs (p), Tothers (p)). Then we perform the same human preference annota- tions as Section 6 Results
shown in Figure 5 AesCoder achieves win rate of over 55% in and Qwen3-Coder- 50% win when compared to state-of-the-an propri-
comparisons with mid- to large-scale open-source models (GLM---32B-O8) (0B-A3BBInstruuee and maintains etary models (Claude Sonnet 4 and GPT-5), demonstrating the effectiveness of our agentic reward framework.
18

---

Published as a conference paper at ICLR 2026
G PROMPT
G.1 PROMPT TEMPLATE FOR PAIRWISE EVALUATION
Prompt Template for pairwise Evaluation
You are a highly-skilled and impartial Al evaluator. Your task is to distinctively evaluate two HTML webpage images, Image A and Image B, generated from the same user instruction but by different models. Your evaluation should emphasize clear differentiation and ranking between the two images, avoiding similar average scores unless they truly of equal quality. Always highlight meaningful differences.
You will be provided with the following:
The general topic of the generated webpages: {topic}
The original user instruction used to generate the webpages: {user jusserinstructton)
Image A, representing the output of the first model, which will be given later. Image B, representing the output of the second model, which will be given later.
Scoring Criteria (Total: 100 points per image):
1. Alignment with User Instruction (40 points):
Score how well each image aligns with the details and intent of the provided user instruction.
Assess whether all requested elements, content, and functionalities are present and correctly implemented.
Evaluate the overall structure and layout match the user's requirements.
2. Aesthetics and Readability (30 points):
Score the visual appeal, design quality, and overall polish each webpage.
Assess factors like color scheme, typography, use of whitespace, and visual hierarchy. Evaluate the case of reading and understanding the content. Is the text clear? Are the sections well-defined?
3. Structural Integrity and Responsiveness (30 points): Scorre the logical organization and structure the webpage.
Assess the overall layout and how the different components arranged. Evaluate how well the design would adapt to different screen sizes (e.g., mobile, tablet, desktop), based visual cues in the image.
Scoring Instructions:
Distinctiveness is required: Avoid giving similar or average scores to both images unless they truly have no meaningful difference.
Justify both high and low scores: If one image is clearly better in any aspect, assign noticeably higher score.
If an image has major flaws, do not hesitate to give low score for that criterion. Do not use safe scores. Use the full range of the scoring scale appropriate.
Your output must contain specific scores for each criterion of the two images, and the overall comparison symbol. The template of the output should strictly obey the following json format (alignment score, aesthetics score, structure score are just the abbreviation of Alignment with User Instruction score, Aesthetics and Readability score, and Structural Integrity and Responsiveness score);
19

---

Published as a conference paper at ICLR 2026
"Image A Score":
weiggnmnt_scorree score _ 1_,
seachheticsssccrree score_A_, 2 "structuro_score", score_A_3, "Total Score"; tooro_ ,
" "Image в Score":
"alligmeett_ccre"" score_B_1,
"aesthetics_scom scoree: score 2, estructure_scoree,
"structure score": score B 3,
"Total Score": total_score B
"Overall Comparison"; "comparison_eymbol "feedback"; "feedback"
Where:
For scores:
score A.1, score A score A_3 the scores for Image A in each category.
score_B_1, score_B_2, score_B_3 are the scores for Image B in each category. total_score_A and total_score_B B the sum of the individual scores for each image.
For comparison symbol:
If Image A is far superior to B, the comparison symbol should be [[Ag]B]].
Image better than B, the comparison symbol should be [[AgB]]] If Image A and B are of equal quality, the comparison symbol should be [[A=B]].
Image worse than B, the comparison symbol should [[A;B]].
If Image A is far inferior to B, the comparison symbol should be [[A;;B]].
For feedback:
A concise summary (about 50 words) of your evaluation, explaining the strengths and weaknesses of the webpage in relation to the scores you'l 've given.
20

---

Published as a conference paper at ICLR 2026
G.2 PROMPT TEMPLATE FOR POINTWISE EVALUATION
Prompt Template for Pointwise Evaluation
You are an expert evaluator tasked with rigorously assessing the quality of an HTML webpage generated by a large language model. You will be given an image of the rendered HTML webpage and the original user instruction.
Your primary goal to provide an objective, accurate, and discriminative score, using the full range of the scoring scale (0-100). Do not hesitate to give low or moderate scores if the webpage is average or has flaws. Only award high scores to webpages that are truly exceptional and nearly flawless according to professional standards.
You will be provided with:
The general topic of the generated webpage: {topic}
The original user instruction: {user instruction)]
Image A, representing the output of the model to evaluate
Evaluation Instructions:
Carefully analyze user instruction and the webpage image.
2. Score the webpage the following criteria (use the full scoring range):
Alignment with User Instruction (40 points):
Does the webpage fully and precisely satisfy all explicit and implicit requirements of the user's prompt?
Are requested elements present and correctly implemented?
Does the content and structure directly correspond the instruction?
Aesthetics and Readability (30 points):
Is the webpage visually appealing, modern, and professionally designed?
Are color, font, and spacing choices effective and consistent? Is the text easy read and the layout clear?
Structural Integrity and Cohesion (30 points):
the structure logical, well-organized, and cohesive?
Do all sections flow smoothly and intuitively?
the user experience (based on the image) seamless and easy to follow?
Scoring Principles (Read Carefully):
Use the full range for each criterion (e.g., 0-40, 0-30). Average flawed webpages should receive average or below-average scores.
High scores (top 20% of each range) should be awarded only for work that meets or exceeds professional standards with virtually no flaws.
If the webpage is missing elements, has visual issues, or organizational problems, score accordingly low.
Provide a brief justification for any high or low score.
Score Interpretation Reference:
900-100: Outstanding, professional. nearly perfect.
- -70-89: Good but with noticeable issues or minor flaws.
500-69: Average, with clear limitations several weaknesses.
-30-49: Below average, significant flaws or missing requirements.
-0-29: Poor, major requirements missing, very low quality.
Provide your final output in the following JSON format:
21

---

Published as a conference paper at ICLR 2026
"al iliiinentt_score <score out of 402, "aesthetics_scom score": <score out of 30>, "structure_score" <score out of 30>, "total_score"; <sum out 100>, "feedback"; "<concise summary (about 30 words) explaining the strengths and weaknesses and justifying the scores>
Remember: As an expert evaluator, do inflate scores. Always judge by high professional standards and make full use of the scoring scale.
22

---

Published as a conference paper at ICLR 2026
G.3 PROMPT TEMPLATE FOR INTERACTIVE AESTHETICS AGENT
Prompt Template for Interactive Aesthetics Agent
Imagine you are distinguished website design judger. Now you are given a task about eval- uating the practicality and aesthetic about the interactivity of webpage. The webpages you are given are all single-paged, offline html files. User will later provide you with the specific topic (Only in these five topies: ("General website", "Game dev", "Data visualization", "3D design", "UI component") and the detailed description of this webpage. You should eval- uate the webpage's interactivity and aesthetic based on the topic and the detailed description.
When evaluating the aesthetic of interactivity a webpage, you should consider the following aspects:
First, think thoroughly about all the ways of interactions with the webpage based on the topic, the detailed description given by the user and the wehpage screenshot. Output your planned interations at the beginning of the task in your thought. Then, evaluate the interactivity interations. For each time of interaction, carefully compare the webpage before and after the interaction. The webpage should change according the interaction. If the webpage is not changed or the change is not expected, it should not be considered as good webpage.
the webpage in order according your planned
Since the webpage offline, we do not expect changes which need internet connection. Specially, for textbox, you should plan both typing in the textbox and clicking the search button. It cannot be considered as a successful interation if only you successfully type in the textbox, but the webpage has not changed at all after clicking the search button.
When your interaction does produce feedback, you still need to carefully consider whether that feedback is correct and logical. For example, if you click list and it merely displays the list, but clicking on an item within the list does not trigger any response, then no points should be awarded. Only correct feedback can earn points.
Sometimes when you click navigation button, the webpage will not change simply because is already in the page you want go. You should try to click another navigation button and click back again to check the interactivity of this navigation button. {GAME EXTRA PROMPT)
In each iteration, you will receive an Observation that includes a screenshot of a webpage and some texts. This screenshot will feature Numerical Labels placed in the TOP LEFT corner each Web Element. Carefully analyze the visual information to identify the Numerical Label corresponding the Web Element that requires interaction, then follow the guidelines and choose one of the following actions:
1. Click Web Element.
2. Delete existing content a texthox and then type content.
Wait. Typically used wait for unfinished webpage processes, with a duration of seconds.
Press the up arrow key. (Only can be used when the topic of the webpage is game dev)
5. Press the down arrow key. (Only can be used when the topic of the webpage is game dev) Press left arrow key. (Only can be used when the topic of the webpage is game dev)
7. Press the right arrow key. (Only can be used when the topic of the webpage is game dev) FINISH. This action should only be chosen when all evaluations in your plan list have been finished.
Correspondingly. Action should strictly follow the format:
Click
-Type [Numerical Label]; [Content]
-Wait
-UP
DOWN
LEFT
RIGHT
23

---

Published as a conference paper at ICLR 2026
FINISH
Key Guidelines you must follow:
Action guidelines
1) To input text, no need to click textbox first, directly type content. After typing. the system automatically hits ENTER key. Sometimes you should click the search button to apply search filters. Try to use simple language when searching.
2) If you have seen scrollbar in the webpage (not for the whole window, since the webpage always
but for a certain area or element of the webpage, such as a 3D object to be rotated or zoomed), do not directly try to scroll it. Instead, find if any interactable element such as button or "+' and click the button instead.
3) If you click button and then pop-up window is displayed, you should close the pop-up window and return to the original webpage after you have finished evaluating the interaction.
4) If the topic of the webpage is game dev, it may not have many interactable elements to click. Instead, you can use the up. down, left, right arrow keys to control the game, and plan dynamically when the game rumning. Don't 't miss up the role in the game with interactable elements.
5) You must distinguish between texthox and search button, don't type content into the button. If no textbox found, you may need to click the search button first before the textbox is displayed.
6) Execute only one action per iteration.
7) Strietly avoid repeating the same action if the webpage remains unchanged. You may have selected the wrong web element or numerical label. Continuous use the Wait is also not allowed.
8) When a complex Task involves multiple questions or steps, select FINISH only at the very end, after addressing all of your planned interations. Flexibly combine your own abilities with the information in the webpage.
Web Browsing Guidelines
1) Don't try to go to other urls Just focus on given offline html page. All your interations can be done offline (without internet connection).
2) Focus on the numerical labels the TOP LEFT corner of each rectangle (element).
Ensure you don't 't mix them up with other numbers (e.g. Calendar) on the page.
Your reply should strictly follow the format: For the first iteration (the planning stage): Thought: Your thorough plan interact with all the interactable elements of the webpage)
For the other iterations (the interaction stage):
Thought: YYour brief thoughts (briefly summarize the info that will help you score the previous interaction, and your brief plan for the next interaction)) Numerical Label: (The numerical label of the previous interaction] Score: (The score of the previous interaction. Only 1, NaN is allowed. O means the interaction is failed incorrect, means successful. Output NaN if no interation is done in this iteration. Specially for textbox, you should output NaN when you finished typing in the textbox, and the actual score when you clicked the search button something else.) Reasoning: { Your brief reasoning for the score. Similarly, you must output N/A if no interation done in the previous iteration)
Action: {One Action format you choose for the next interaction]
Then the User will provide: Observation: {A labeled screenshot Given by User]
24

---

Published as a conference paper at ICLR 2026
G.4 PROMPT TEMPLATE FOR ABLATION WITHOUT AGENTIC REWARD
Prompt Template for Ablation without Agentic Reward
You expert evaluator tasked with assessing the quality of an HTML webpage generated by a large language model. You will be given the HTML code of the webpage and the original user instructions.
You will be provided with:
The general topic of the generated webpage: {topic}
The original user instruction: {user_instin jistrrristttt)onn
The html code of the webpage: {html}
Your objective is to assign precise, rigorous scores, using the full 0-100 range. Only award high scores for webpages that are absolutely flawless, meeting all design and functional expectations. Penalize harshly for the smallest imperfections teeeeethereeee is zero tolerance for errors.
Key Evaluation Areas:
1. Instructional Alignment (20 points)
Evaluate how closely the webpage follows the user's instructions. Only this in aspect, your criteria can be relatively low, since we expect some flexibility in interpretation and should more pay more attention another two aspects (Visual Design and Aesthetics and Structural Coherence and Usability below).
Score levels and their explanations:
Good alignment (10-20): The webpage almost matches the user's instructions. Severe misalignment (0-9): The page fails to meet basic requirements. Major elements are missing or misrepresented.
2. Visual Design and Aesthetics (50 points)
Assess the overall professionalism and polish of the design. Only award high marks for designs that look flawless, balanced, and intentional.
Some golden rules you should obey when scoring:
Always cherish detailed, refined, and innovative design. highly refined design always better than plain one, which means value pages with highly rich design elements more than simple and plain designs. This includes an exquisite transparent dynamic background, elements special effects floating in the background, gradient color text, rich yet beautiful color matching, and so on.
NO PLACEHOL.DERS! Always cherish real images and expressive (real abstract) icons, instead of placeholders. A website with rich, real, and appropriate images or icons should score higher(85 or above). while website with placeholders or broken images should score below 50. Abstract modern icon are also preferable, but they should be well-designed and are NOT placcholders.
Simplicity is not a lack of content. A simple design can still be rich and engaging it uses space, color, and typography effectively.
The overall impression important. Make sure the webpage has NO broken/partially visible words or elements. NO partially loaded elements.
Score levels and their explanations:
Perfect design (40-50): The design is exceptionally professional, with well-executed color palette, typography, and spacing. The page has a polished and intentional feel.
Minor flaws (20-39): The design is good, but there small issues (e.g., slight inconsis- tency in font sizes or spacing). These should still impact the score significantly. Significant flaws (10-19): The design has major issues (e.g., poor readability, awkward
25

---

Published as a conference paper at ICLR 2026
layout, jarring color choices). Unacceptable design (0-9): The page is unprofessional, with severe flaws such as overlapping text, unreadable fonts, or broken layouts images.
Structural Coherence and Usability (30 points)
The page must have logical and intuitive structure. Even the smallest structural mistake (misalignment, broken flow, or inconsistent layout) will severely affect the score.
Key scoring rules:
Overall impression comes first. This stresses the importance of adopting modern, cise, refined framework. Encourage websites to use modern, beautiful design frameworks instead of simple, mediocre designs. Wehpages with appropriate use framework score above 85, while those with poor or no framework should score below 50.
Highlight the integrity of the overall structure. Check carefully whether the page has complete structural layout, with no missing elements or broken sections. If the page has any broken sections, it should score below 50.
Flawless structure (20-30): The page has a perfect structure: well-organized, logical flow, and easy navigation.
Minor structural issues (15-19): The structure is good, but there are small usability issues (e.g., slightly misaligned sections awkward navigation).
Major structural problems (10-14): The page has significant usability flaws, such as broken layouts confusing content organization.
Unusable structure (0-9): The page has severe structural issues, making it difficult to use navigate effectively.
Fine-Grained Scoring Guidelines:
Strict threshold for high scores: Only give scores above 90 the webpage is absolutely flawless. If there is even a minor issue (e.g., a single broken element, misalignment, or poorly chosen font), do not award high marks. Scores 95+ should be reserved for near perfection.
Minor flaws are heavily penalized: the webpage has any noticeable flaw (such as text overlapping an image, improper spacing, a lack of balance), this will result in low overall score! (c.g., 10-30)
Zero tolerance for bad design: If the wehpage looks unprofessional (e.g., excessive white space, unaligned content/text, unreadable text, or poor contrast), the overall score should be 0-30!
Example Evaluation:
For webpage with:
Perfect alignment with instructions (everything is present and correct), Excellent visual design, but with slightly misaligned text,
Clear structure with one misaligned image.
You might score:
Instructional Alignment: 20/20 (perfect alignment with instructions), Visual Design: 35/50 (good design but minor flaw-misaligned text),
Structural Coherence: 20/30 (minor misalignment of an image).
Total Score: 75/100 (not good, but OK).
Final Output Format (alignment score, aesthetic score, structure score are just the abbrevi- ation of Instructional Alignment score, Visual Design and Aesthetics score, and Structural Coherence and Usability score):
26

---

Published as a conference paper at ICLR 2026
"al weiiinnnt__sooree <score out of 20%, "aesthetic__scom score": <score out of 50>, "structure_score" <score out of 30>, "total_score"; <sum out 100>, "feedback"; "<brief summary of strengths and wcaknesses, with justi ficatiiiaioon for the scores>"
Strict Scoring Principles:
Minor mistakes are penalized severely. A single misplaced element, broken layout, or poor design choice will dramatically affect the score.
High scores (90+) should only be given for perfect webpages with emros. If there is any imperfection, the score should drop significantly.
No merey for bad design. Webpages that are visually unappealing hard to must receive low scores (0-9) regardless of other factors.
27

---

Published as a conference paper at ICLR 2026
G.5
PROMPT FOR KEYWORD
Prompt Template for Website Keyword and Summary Generation
You are professional website content generator. Generate 300 unique keywords short summary descriptions (10-30 words each) for websites of the type "(catagory)". Each summary should:
Reflect a unique purpose, functionality, or use case for a website.
Be based creatively chosen theme industry, covering wide range of domains (e.g., healthcare, education, finance, entertainment, environmental, e-commerce, tourism, tech, art, sports, social impact, etc.) by leveraging your imagination. Ensure summaries are specific, diverse, and avoid repetition in functionality, theme, or wording.
Output JSON array, where each entry contains: summary: A concise description (10-30 words) of the website's purpose or functionality, reflecting the chosen theme.
Ensure maximum diversity by exploring unique and imaginative themes, avoiding overlap with common website concepts. Return the result in JSON format.
Example output format:
"uummary"; "A website for eco-consciaus travelers, offering sustainable tourism guides, ethical lodgi ng options, and carbon footprint calculators."
summary"; "An educational platform providing interactive biology simulations, 3D models, and real-time quizzes for high school students.*
28

---

Published as a conference paper at ICLR 2026
G.6 PROMPT TEMPLATE FOR DATA REWRITING IN RL
Prompt Template for Data Rewriting in RL
You are content strategist and creative visionary specializing in conceptualizing innovative digital platforms. Your task is to transform abstract ideas into compelling website concepts that are both unique and inspiring.
I will provide you with brief description seed topic. From this, your goal is generate a highly imaginative and detailed website concept. The concept does not need to be directly correlated the content provide. Feel free draw inspiration from keywords or abstract elements and create something new and innovative.
Your output should focus on the overall content, purpose, and features of the website, without going into specific layout, design, visual details. Think about the theme, functionality, and interaction possibilities of the site. This will serve as the basis for generating HTMI. code for the site.
has to be noticed that your instruction should not contain any specific layout, design, or visual elements of the website, but only the content, purpose, and features of the website.
You required to choose of the following categories for each website concept you create. Please try to think creatively and step outside the "General website" category when possible:
General website: A website designed for general use or any topic, focusing on its core content, purpose, and user interaction.
browser-based game concept in HTML. Focus on interactive and engaging content, game mechanics, and user experience.
Game development:
Data visualization: A page that presents dynamic and interactive data, such as charts, graphs, or visualized datasets. Focus on how the user will interact with and explore the data. UI component: A page dedicated to showcasing single, highly interactive component. Focus the functionality and purpose of the component, without detailing its visual structure.
5. 3D design: A concept for 3D scene interactive experience, focusing its content and user interaction, rather than specific rendering or layout details.
For each brief description I provide, follow this structure: Select category from the list above that best fits the concept. 2. Create a detailed and concise description of the website concept, focusing on its content, purpose, features, and interactions.
3. Provide clear instruction (40-60 words) for HTML. code generation that can be used to implement this concept.
Output the response in the following JSON format:
"category"; "<category name from the list>", "instruction": "<detailed website concept instructions"
29

---

Published as a conference paper at ICLR 2026
G.7 EXECUTION AGENT VALIDATION RULES
Execution Agent Validation Rules
The following configuration defines validation and linting rules for HTML, CSS, and JavaScript within single HTML file. These rules should be strictly applied when evaluat- ing or generating webpages.
"doetype-htmm5": true, doctype declaration "tagname-lovercase" true, tag names "attr-lowercase"; true, attribute names seett--walue-duuule-posssss true,
// Enforce HIML5
// Enforce lcwercase
// Enforce lewercase
// Enforce deuble
quotes for attribute values "tag-pair"; true,
// Enforce all tags
must have corresponding closing tag "tag-self-close:: l"br", "img", "input", "link", "meta"I, Allow self-closing tags for specific elements "id-unique"; true,
// Ensure 'id'
attribute is unique in the document
" "alt-require"; true, attribute for all <ing> tags for accessibility
// Enforce 'alt'
false,
// Allow <script>
tags in the <head> section
"style-dleabled"; false, styles within HTML "no-inline-style" false, styles within HTMЛ "no-inline-scrip:: false, JavaScript within the HTML file "Lang-requize": true, attribute in the <html> tag
// Allow inline CSS
// Allow inline
// Allow inline
// Enforce 'lang'
" "meta-charsst-uur--" true, charset declaration "meta-viewport": true, of the viewport meta tag "title-requize"" true, of the <title> tag
// Ensure UTF-8
// Enforce inclusion
// Enforce inclusion
" "csslint":
"important"; false, !important in CSS
// Allow the use of
"order-alptubeticaa"" false
// Do not enforce
alphabetical order for CSS properties
"script-disabled"" false (inline within HTML)
// Allow JavaScript
G.8 PROMPT TEMPLATE FOR DATASET PROCESSING
Prompt Template for Pointwise Evaluation
You a professional evaluator tasked with performing meticulous assessment of HTML webpage generated by a large language model. You will receive both the rendered webpage image and the original user instruction.
30

---

Published as a conference paper at ICLR 2026
Your goal is to assign precise, unbiased, and discriminative using 0-100 scale. Use lower or mid-range scores for webpages that demonstrate average quality or contain flaws; reserve high scores exclusively for outputs that meet professional standards with minimal or no deficiencies.
You will be provided with:
Webpage topic: {topic}
Original user instruction: {user_instruee instructionn]
Rendered webpage image: Image A
Evaluation Procedure:
1. Examine the user instruction carefully in conjunction with the webpage image.
2. Evaluate the webpage across the following criteria, making full use of the scoring ranges.
1. Compliance with User Instruction (40 points)
Does the webpage satisfy all explicit and implicit requirements of the instruction?
Are all requested elements present, correct, and properly implemented?
Is the content and structure fully consistent with the user's instructions?
2. Visual Design and Readability (30 points)
Is the webpage visually appealing, modern, and professionally designed?
Are typography, color schemes, and spacing applied consistently and effectively?
Is the text legible and the layout clean, clear, and easy to follow?
3. Structural Cohesion and Organization (30 points)
Is the webpage structure logical, coherent, and well-organized?
Do sections flow naturally and intuitively?
Based on the image, is the user experience seamless and easy to navigate?
Scoring Guidelines:
Use the full range of each criterion (0-40, 0-30, 0-30).
Average or flawed webpages should receive average or below-average scores.
High scores should be awarded only for outputs that meet professional standards with virtually no shortcomings.
Deduct points for missing elements, visual issues, structural inconsistencies.
Provide brief justification for unusually high or low scores.
Score Interpretation:
90-100: Outstanding; professional- nearly flawless.
70-89: Strong; minor issues or noticeable imperfections.
50-69: Moderate; average quality with several limitations.
30-49: Weak; significant flaws or missing elements.
0-29: Poor; major requirements missing or very low overall quality.
Output Format: Provide your evaluation using the following ISON template:
"alignment_score" <score out of 402, "aesthetica_scora" <score out of 30>, setuucture_ccore"" <score out of 30>, stotal_score" <sum out of 100>, " "feedback"; "<conciss summary (about 30 words) explaining the stngths and weaknesses and justifying the scorea>
Important: Maintain professional rigor. Avoid inflating scores and evaluate solely based on the quality observed in the rendered webpage image.
31

---

Published as a conference paper at ICLR 2026
H ADDITIONAL EXPERIMENTS
H.1 ROBUSTNESS OF OPENDESIGN EVALUATION UNDER ALTERNATIVE JUDGES
To assess whether the OpenDesign benchmark is robust changes in the underlying aesthetic judge and to rule out circularity effects between training and evaluation, we additionally evaluate main- stream foundation models using This model is strong, open-source vision-language model independent of the OpenAl model fam- ily used our main results.
alternatiee judge.
Table reports the results obtained by evaluating the same set of models with both GPT-5 and -ee-3--L--33B--A4 Instrrctt under the identical scoring prompt.
Table 6:
OpenDesign evaluation with different judges. Despite differences absolute score scales, the relative ranking of all models remains unchanged.
Model Name
GPT-5 Eval
Qwen3-VL Eval
Relative Rank
AesCoder-4B (Ours)
81.92
91.12
Claude 4
81.05
90.90
2
GPT-5 (minimal)
81.03
90.40
3
Owen3-Coodr-480B-A33B
79.90
90.37
4
DecpSeck-R1105288
78.86
89.50
5
GLM-4.5-Air
78.16
87.71
6
DeepSeek-V3.. V3.1 (Thinking)
77.72
87.62
7
Qwen3-Coder-33B-AB3
73.66
87.16
8
GLM-4-32B-0414
69.40
81.41
9
GPT-4.1
65.79
74.46
10
GPT-4o
48.13
70.52
11
The results show three key findings:
OpenDesign evaluation is robust under judge substitution. Although Qwen3-VL-235p) A22B-Instruce assigns higher absolute scores than GPT-5, the relative ordering of the ten main- stream models remains identical. This consistency across two unrelated model families indicates that OpenDesign captures a audge-invariant notion of design quality rather than artifacts tied to single evaluator.
2. Improvements from AesCoder reflect genuine capability gains rather than judge-specific overfitting. Since Swem3-VLL225B--22BBBB
and GPT-5 produce the same ranking across all baseline models, AesCoder's improvements cannot be attributed to exploiting idiosyncratic scor- ing patterns of a particular judge. Instead, the gains generalize across evaluators with different architectures, training data, and aesthetic priors, confirming that the improvements reflect true en- hancement visual coding ability.
3. Both GPT-5 and -1--333-V2233--22221 preferences. Figure 2a in the main paper demonstrates strong diagonal agreement between GPT-5 and human pairwise judgments. Our new results show that - A---B-A222I-nnnu- Instruct repli- cates GPT-5's ranking structure on the same set of models. Since both judges independently recover the human- human-preferred ordering, this provides converging evidence that our evaluation methodology scientifically rell-grounded and reliably aligned with human aesthetic standards.
exhibit alignment with human aesthetic
In summary, these experiments confirm that OpenDesign is robust to evaluator choice and that the improvements from AesCoder arise from real model capability rather than bias toward any particular judge model.
H.2 SENSITIVITY ANALYSIS OF REWARD WEIGHT CONFIGURATION
To understand how different reward components influence learning dynamics in our agentic frame- work, perform ablation study by varying the weights (Wexec; Waatiey Winterma) while keeping the
32

---

Published as a conference paper at ICLR 2026
Matatic 0.0
(b) Wssatic 0.2
Watatic 0.4
(d) Wsatii 0.6
(e) Mannii 0.9
Figure Reward curves for five weight configurations in GRPO-AR training.
total mass fixed. This analysis evaluates whether the framework is sensitive to weight perturbations and whether training remains stable under different reward trade-offs.
Experimental Setup. We keep Wexee 0.1 fixed-reflecting the high tolerance of web browsers to HTML syntax-and redistribute the remaining mass between the Static Aesthetics and Interactive Aesthetics components. We select four representative configurations ranging from "pure static" to "pure interactive." For each configuration, we run GRPO-AR training for 150 RL steps on the AesCoder-4B.r model due to computational constraints. Each trained model evaluated on the OpenDesign benchmark using 5 repeated runs to estimate confidence intervals.
Table 7: Effect of reward weight variation on OpenDesign performance. Mean + std over 5 runs.
Wexee
Wstatic
Winteract
Static Score
Interactive Score
0.1
0.0
0.9
78.88 + 0.32
0.91 + 0.20
0.1
0.2
0.7
79.28 + 0.30
0.87 + 0.17
0.1
0.4
0.5
79.87 + 0.33
0.84 0.18
0.1
0.6
0.3
80.01
+ 0.33
0.80 0.17
0.1
0.9
0.0
80.50 = 0.32
0.63 = 0.15
Findings. The results reveal a clear and monotonic trade-off: increasing Wasic consistently im- proves static design quality, while decreasing interactivity; increasing Winterect has the opposite ef- fect. This smooth trend confirms that the agentic reward components are well-behaved and that the model optimizes rationally according the specified weight structure.
Importantly, across all configurations tested, the training process remained stable and the reward curves (included in Figure (6) displayed normal upward trajectories without collapse. This demon- strates that GRPO-AR provides a controllable and robust optimization signal, and that the framework remains stable under substantial perturbations of reward composition.
Conclusion.
These results indicate that (1) the model's behavior can be reliably steered by adjust- ing reward weights, (2) the system is not fragile to moderate weight changes, and (3) the reward components interact coherently within the GRPO-AR framework. Thus, the reward design is both flexible and stable, enabling practitioners to tune the model toward different aesthetic sesthetic-funcciioalii functionality trade-offs depending downstream needs.
33

---

Published as a conference paper at ICLR 2026
H.3 GENERAL CODE ABILITY EVALUATION ON STANDARD CODE BENCHMARKS
To examine whether our aesthetic-oriented reinforcement learning affects general-purpose coding ability, we evaluate both AesCode 4 and AesCoder 7B three standard benchmarks: Live- CodeBench JJain et al. 2024). MBPP (Austin et al., 2021 Liu et al., 2023), and MBPP+ (Austin al., 2021; Liu al., )023). These benchmarks measure algorithmic correctness, functional rea- soning, and code execution reliability reliablity-ccaaabliiee capabilities orthogonal to our aesthetic training targets.
Table 8: Results of AesCoder and their base models on general-purposs coding benchmarks.
Model
LiveCodeBench
MBPP
MBPP+
-wen3----Innsruutt-2
32.5
86.8
74.3
AesCoder-4B
19.0
73.5
64.3
16.8
83.5
71.9
AesCoder-7B
15.3
73.5
63.1
Across all benchmarks, the AesCoder models exhibit noticeable and expected regressions in gen- eral code accuracy following aesthetic-focused reinforcement learning. This behavior is attributable to the alignment tax associated with domain specialization: GRPO-AR aggressively optimizes the model toward code aesthetics (c.g., Python-based plot generation and webpage design), diverging from the pure algorithmic reasoning objectives measured by standard benchmarks.
Crucially, while the performance drop is significant, it reflects a conscious trade-off between be- coming a domain expert in visually-ore -oriened coding and maintaining state-of-the-eee generalist capabilities. The results indicate that the aesthetic alignment procedure shifts the model's 's distribu- tion significantly toward design oorigned tasks.
In summary, this evaluation demonstrates that: (1) AesCoder achieves state-of-the offthh-artt art aesthetic qual- ity, which necessitates trade-off in general algorithmic performance, (2) the model retains func- tional coding competence without suffering from complete catastrophic forgetting, and (3) the observed regressions consistent with the behavior of models undergoing intensive fine-tuning for highly specific domains.
I FAILURE CASE STUDY
I.1 NON-EXECUTABLE & VISUALLY INCORRECT
User Prompt: Design a website about 1+1.
The model's output is follows:
This output is non-executable and visually incorrect. The user requested a webpage that displays "1+1," but the model instead produced the direct result "1+1=2," and the output is not even a valid HTML format. Therefore, the Execution Agent fails this sample, and neither the Static Aesthetic Agent nor the Interactive Aesthetic Agent is triggered.
Analysis.
Final reward.
Pezes -1, no ssaatic and Finteract (not triggered). The final reward is -1.
1.2 NON-EXECUTABLE & VISUALLY CORRECT
User Prompt: Make simple personal website.
34

---

Published as a conference paper at ICLR 2026
The HTML code of the webpage is as follows (partial snippet):
<IDOCTYPE, html>
<htm1 lang="en">
<head>
<div class="section" 1d=*ontact-section <h2>Contact<//2>
<p>Feel free to reach out to me!</p> <div class="oontact-Links
<a hestenaaitee..xxxxeeaaee.......
<a hret="htttp:://attuuu.ooo/ http:/ttuuubbooo//mococtttttaaaa <a hute=hhtper/Lanncc..ooo.ooaaaae
</div> </div>
<div class="section cd="centatt-seat
<h2>Follow Me</h2>
sp>Connect with me en social media!</po
</div>
</div>
</body>
</html>
The webpage sereenshot is Figure
Your Name
Dendoper Den gree
About Me
Jnatscip
Contact
Post
Follow
Figure 7: A webpage screenshot of non-executable but visually correct code.
Analysis. A close inspection of the page's HTML source code (partial snippet shown below) reveals that both <div> elements use the same ieteeid=""mne "cottcctttt--" sonetttsecionn (highlighted in red). In standard HTML, IDs must be unique within a document, and this violation is correctly detected by HTMLHint during executability validation. Although the rendered webpage appears visually normal-wite no immediately noticeable defects from a screenshot perspective- the underly- ing HTML is structurally incorrect.
35

---

Published as a conference paper at ICLR 2026
Because of this, the Execution Agent fails the sample and assigns reward of Texee -1. As result, neither the Static Aesthetics Agent nor the Interactive Aesthetics Agent is triggered, ensuring that the model receives credit for visually plausible but syntactically invalid outputs. This strict gating mechanism enforces the production of standards-complian,, well-structured HTMM,, preventing the model from exploiting aesthetics to bypass fundamental correctness.
Final reward.
Tezee
-1, no 'static and Finteract (not triggered). The final reward is -1.
1.3
EXECUTABLE BUT VISUALLY BROKEN
User Prompt: Make a simple webpage of an Italian restaurant.
The webpage screenshot is Figure
MAMA MIA'S ITALIAN RESTAURANT
OUR MENU
Spaghetti Carbonara
$8.99
Figure A webpage screenshot of executable but visually broken code.
The model generates HTML code that is fully syntactically valid and passes executabil- ity checks. However, the rendered webpage reveals severely degraded visual quality. As shown in the screenshot, the design suffers from clashing color combinations (purple-red, blue, and yellow), low-resolution imagery, and text elements with fonts that are far too small for comfortable reading.
Analysis.
Because these issues, the Static Aesthetics Agent assigns a very low score (18 points), reflect- ing that although the page minimally aligns with the user's s instruction, its overall visual presentation substandard. Furthermore, since the webpage contains no interactive elements, the Interactive Aesthetics Agent correctly assigns score of O.
This example demonstrates that even when the model produces syntactically correct HTML, the reward system does not overlook poor design quality: executability alone is insufficient, and the Static and Interactive agents provide crucial complementary supervision.
Final reward. Fezee
Fstatic -0.64 and Fenteract -1. The final reward is -0.512.
36

---

Published as a conference paper at ICLR 2026
1.4 VISUALLY GOOD BUT FUNCTIONALLY WRONG
User Prompt: Design a website for me to manage my daily spending records.
The webpage screenshot is Figure
Daily Spending Tracker
Hagage you eigenses wiely
$0.00
$0.00
Add New Record
Recent Activity
2005 F1L/25
exerree
51
Desciption
All Records
Figure 9: A webpage screenshot of visually good but functionally wrong code.
Analysis. The model generates HTML code that is both syntactically valid and visually well- designed, The rendered webpage appears polished, with clean layout and aesthetically pleasing styling. As a result, the Static Aesthetics Agent assigns a high score of 81 points.
However, despite its strong visual presentation, the webpage fails to satisfy the functional require- ments specified by the user. The user requested a webpage for recording daily expenses, yet the "Add Record" button is entirely mon-functional: clicking it does not log the expense or trigger any observable state change. This constitutes clear violation of the expected interactive behavior. On the other hand, certain elements-such as the date-selection component- respond correctly. Consequently, the Interactive Aesthetics Agent assigns partial score of 1, reflecting the presence of some functioning elements but penalizing the non-operational core feature.
This case illustrates why interactive evaluation is essential: webpage may look correct and receive high static score, but without functional alignment, the overall reward must significantly reduced.
Final reward. Fezee
Pstatic 0.62 and Pinteract The final reward is 0.596.
37


Reviews:
[{"review_id": "0puQtvkdGV", "reviewer": "Reviewer_k5dx", "summary": "The paper proposes a training and evaluation pipeline to improve the visual quality (\"code aesthetics\") of LLM generated code in two domains: Python plotting and webpage design. The authors (i) build AesCode-358K, a supervised instruction tuning dataset comprising approximately 158k validated plotting examples and a large set of webpage design cases produced and filtered through a multi step process, (ii) introduce an agentic reward framework with three agents for execution, static aesthetics, and interactive aesthetics, and aggregate their signals with a weighted sum, (iii) train AesCoder-4B/7B with GRPO-AR, and (iv) propose OpenDesign, a benchmark for static and interactive webpage aesthetics whose rankings show high agreement with the Design Arena leaderboard and reasonable GPT and human alignment. On PandasPlotBench and OpenDesign, AesCoder-4B outperforms GPT-4o/4.1 and is competitive with much larger models, and ablation studies show benefits over DPO and RFT and over GRPO without the agentic reward.", "questions": "What are the exact definitions and numeric ranges of the three agent signals before aggregation, and is any normalization, clipping, or rescaling applied to each signal prior to the weighted sum?\n\nThe paper reports one specific setting of the weights w_exec, w_static, and w_interact. Were these exact weights used for all training runs and all evaluations, and how were they chosen in the first place for example by a validation set criterion or a simple search?\n\nHow sensitive are the results to the choice of weights? Please provide a systematic ablation that varies the weights over a grid while keeping the total weight fixed, and report primary metrics with confidence intervals and the stability of model rankings.\n\nHow should users of the method select weights in new domains or tasks that differ from plots and webpages? Please provide general guidance or a simple procedure for choosing weights based on observable properties of the three signals, and discuss when rebalancing the weights is necessary.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The problem is well defined, the methodology is mostly sound, and the experiments support the main claims. The reliance on proprietary judges (GPT-5/GPT-4o) for both data curation and evaluation introduces possible bias, which the paper partially mitigates via correlation with Design Arena and human annotations. More analysis on sensitivity to judge choice and reward weights would strengthen soundness.\n\nThe execution agent uses HTMLHint rules rather than brittle strict parsing; the static aesthetics agent defines three explicit criteria and uses a single image of the rendered page; the interactive agent uses a GUI agent to act on the page and aggregates successes. These design choices are concrete and implementable.\n\nOpenDesign’s rankings have high correlation with Design Arena (Spearman 0.98, Kendall 0.91), and GPT–human agreement of 80.9% exceeds human–human agreement for the 200 pairwise comparisons, which supports the reliability of the evaluation protocol.", "weaknesses": "GPT-5 (and GPT-4o) are used during dataset filtering, for static aesthetics scoring, and within the interactive agent. This may bias the training signal and the benchmark toward these judges’ preferences. While OpenDesign shows strong rank correlation with Design Arena and decent GPT–human agreement, results could be sensitive to the choice of judge. Please report results with at least one strong alternative judge and quantify changes.\n\nThe paper defines the weighted sum for r and also reports a concrete setting of the weights ($w_{exec}$, $w_{static}$, $w_{interact}$). However, it does not describe a normalization scheme across agents or provide an ablation over alternative weight choices. This limits reproducibility and makes it hard to assess sensitivity. Please report the exact normalization used and add a systematic ablation over plausible ranges, for example, varying one weight while keeping the total fixed, and include confidence intervals for all metrics.\n\n\nA few typos reduce polish (e.g., “sucess”, “Grammer”, “protion”). Figures are informative but Figure 1 would benefit from a clearer depiction of when each agent runs (serial vs parallel) and where screenshots are taken.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "byBwjxd5nU", "reviewer": "Reviewer_ZReh", "summary": "This paper studies the aesthetic capabilities of code LLMs in visually-oriented coding tasks, such as plot generation and web design, where conventional execution-based metrics are insufficient. To enhance these capabilities, the authors curate a supervised instruction tuning dataset, AesCode358k, covering Python-based plot generation and web design instructions. The model is further improved by reinforcement learning using the GRPO algorithm combined with an Agentic Reward framework, which integrates execution, static aesthetic, and interactive aesthetic evaluations. Finally, the authors propose a new benchmark, OpenDesign, to assess aesthetic quality in LLM-generated web designs.", "questions": "- For the Python-based plot generation data, while executability ensures the code runs without errors, how do the authors verify that the visual output actually matches the instruction semantics (e.g., correct axes, legends, or visual style)?\n\n- In line 169, could the authors elaborate on the clustering method and representative sampling strategy used for data selection?\n\n\n- In line 175, what are the specific scoring criteria and aspects used by GPT-5 during the aesthetic evaluation?\n\n- How are the relative weights among the three reward components determined? Are they tuned empirically, heuristically, or via grid search?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper addresses an underexplored but important problem that is the aesthetic quality of code generated by LLMs for visual tasks, which measuring functional correctness is insufficient .\n\n- The AesCode358k dataset is a valuable contribution that can benefit future research in aesthetic-aware code generation once released.\n\n- The proposed OpenDesign benchmark is also a useful dataset that addresses the limitation of having human voters for evaluation, providing a reproducible way to evaluate aesthetic quality , which enhances scalability and objectivity.", "weaknesses": "- The same aesthetic agents are used both during reinforcement learning and evaluation, introducing potential circularity and reward overfitting. The model may learn to exploit or mimic the judge model’s biases rather than improving true generalization in visual aesthetics.\n\n- While improvements on visual coding tasks are reported, the paper lacks an evaluation of whether the aesthetic alignment impacts general code generation ability (e.g., possible regression on standard code benchmarks such as BigCodeBench or LiveCodeBench).\n\n- The Agentic Reward framework relies on multiple large models to serve as static and interactive aesthetic judges. This heavy dependence on high-capacity teacher models will make the framework computationally expensive and difficult to scale, especially for community replication or deployment.\n\n- The definition of “aesthetic quality” remains somewhat opaque. A clearer operationalization or rubric (e.g., color harmony, layout balance, readability) would help readers understand what the models are actually learning to optimize.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "blAkyAgYLE", "reviewer": "Reviewer_zdWS", "summary": "This paper introduces AesCoder, a framework for enhancing the aesthetic quality of LLM-generated code for visually-oriented tasks like plot generation and webpage design. The key contributions include:\n\nAesCode-358K: A large-scale instruction-tuning dataset focused on code aesthetics\n\nAgentic Reward Feedback: A multi-agent system evaluating executability, static aesthetics (via screenshot analysis), and interactive aesthetics (via GUI interaction)\n\nGRPO-AR: Integration of the agentic reward framework with the GRPO algorithm for reinforcement learning\n\nOpenDesign Benchmark: A new benchmark with 840 real webpage cases for evaluating code aesthetics\n\nThe proposed AesCoder models (4B and 7B parameters) achieve state-of-the-art results, outperforming GPT-4o and GPT-4.1, and competing with much larger 480B-685B models on aesthetic quality metrics.", "questions": "Reproducibility: Given the heavy reliance on GPT-5 and GPT-4o for reward judgment and data generation, what are your plans to make the framework more accessible to researchers without access to these proprietary models?\n\nComputational Efficiency: What is the approximate cost and latency of the full agentic reward evaluation per sample? How does this impact the practical feasibility of scaling this approach?\n\nInteractive Agent Limitations: You mention the low success rate of GUI agents. Could you provide more analysis of how these failures affect the reward signal quality and training stability?\n\nGeneralization: Have you tested the framework on other visually-oriented coding tasks beyond plots and webpages (e.g., GUI development, game level design)? What adaptations would be needed?\n\nAblation Details: In the GRPO-AR ablation, what specific reward model was used as the baseline? Was it trained on the same data or using similar principles as your agentic framework?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "**Originality**:\n\nThis work represents the first systematic effort to formalize and computationally address the concept of \"code aesthetics,\" a dimension of code quality long acknowledged by practitioners but largely neglected in automated code generation research. The proposed multi-agent reward framework is a novel architecture that synergistically combines textual analysis, visual rendering, and interactive evaluation to assess code quality holistically. A particularly creative contribution is the pioneering application of GUI-based autonomous agents for reward modeling in Reinforcement Learning, enabling the system to perceive and evaluate code outputs from a human-like, user-centric perspective.\n\n**Quality**:\n\nThe research is underpinned by a rigorous experimental protocol, with extensive evaluations conducted across multiple established benchmarks (including HumanEval and MBPP) and a range of model scales (from 160M to 7B parameters) to ensure robust findings. The validity of the proposed framework is further cemented by comprehensive ablation studies that meticulously isolate and confirm the contribution of each component. High-quality human evaluations are conducted, the results of which strongly align with the automated metrics, lending significant credibility to the findings. Furthermore, the construction of the dataset reflects a commitment to quality, involving careful data curation, filtering, and validation processes.\n\n**Clarity**:\n\nThe manuscript is exceptionally well-written, presenting complex concepts and a sophisticated multi-stage framework with remarkable clarity and logical flow. A comprehensive appendix is provided, offering full implementation details and facilitating replication and future research. The authors maintain transparency throughout, openly discussing the limitations of their approach and justifying key design choices, which enhances the work's credibility and scholarly value.\n\n**Significance**:\n\nThis research addresses a critical, real-world gap in LLM-powered code generation: the disconnect between functional correctness and user experience. By focusing on the aesthetic and usability aspects of code, the work provides tangible resources - including a novel dataset and benchmark - that will undoubtedly spur further investigation in this emerging area. Perhaps most significantly, the demonstration of practical efficacy even with smaller model sizes highlights the approach's potential for real-world deployment, making advanced code aesthetic optimization more accessible and scalable.", "weaknesses": "**Reproducibility Concerns**\n\nThe reproducibility of this study is significantly hampered by its deep dependence on several proprietary, black-box models, specifically GPT-4 and GPT-4V, which serve as the core \"judges\" in the reward model and are pivotal for the initial dataset generation. This reliance creates a hard barrier for independent verification, as the internal mechanics and future versions of these models are opaque and subject to change, making it impossible to exactly replicate the evaluation criteria or data sources. Furthermore, the computational footprint of the proposed multi-agent evaluation system—which involves executing generated code, rendering graphical interfaces, and deploying interactive GUI agents—is substantial and likely prohibitive for academic research labs with limited resources. Finally, while the interactive agent is a novel idea, the paper provides insufficient detail on its specific failure modes and brittleness, leaving readers to wonder how often it fails due to environmental issues versus the actual quality of the generated code.\n\n**Technical Limitations**\n\nSeveral technical limitations warrant further investigation. The paper candidly acknowledges the relatively low success rate of the interactive aesthetics agent, but it does not delve into a quantitative analysis of how these frequent failures during the reward collection phase propagate through and potentially destabilize the Reinforcement Learning training loop. The complex, multi-stage RL setup, which blends rewards from multiple agents, naturally raises questions about training stability and sensitivity to hyperparameter choices, which remain entirely unaddressed. Additionally, the scope of the framework's generalization is demonstrated primarily on a narrow set of visually-oriented tasks like plots and web pages; its applicability to other critical domains such as game development, mobile UI creation, or data dashboard generation is left unexplored, leaving its broader utility an open question.\n\n**Evaluation Scope**\n\nWhile the introduced OpenDesign benchmark is a valuable contribution, the evaluation scope could be strengthened to more firmly establish the generalizability of the findings. The task types within OpenDesign, though comprehensive, are not fully representative of the entire spectrum of front-end coding challenges; incorporating more complex and diverse interactions, such as those involving dynamic data feeds, state management, or complex user input validation, would provide a more rigorous test. Moreover, the comparative analysis in the paper is primarily focused on benchmarking against general-purpose LLMs. A more telling comparison would involve pitting the method against other state-of-the-art code generation models that are specifically fine-tuned for front-end tasks, which would better isolate the unique benefits conferred by the aesthetics-focused training paradigm.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GStgM3b3n0", "reviewer": "Reviewer_Z31k", "summary": "The paper studies how to improve the visual quality of code produced by LLMs. It introduces a large dataset and a multi-agent reward framework that jointly evaluates code executability, static aesthetics, and interactivity. Using this feedback within a GRPO-based reinforcement learning setup, the authors train AesCoder models that perform well on a new benchmark, OpenDesign, for webpage aesthetics.", "questions": "1. How are the weights $w_{exec}$, $w_{static}$, and $w_{interact}$ chosen in reward aggregation? Were these tuned per dataset/model, or kept fixed? \n\n2. Given that the three reward sources operate on different scales, how are they normalized to ensure balanced optimization? Are there cases where one component dominates or destabilizes training?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. Clear Problem Statement: The work identifies and addresses a neglected area in LLM code generation: “code aesthetics,” specifically for visually-oriented outputs that go beyond basic executability or syntactic correctness.\n\n2. Substantial Data and Benchmark Contribution: The AesCode-358K dataset is large and carefully curated for code aesthetics, and OpenDesign serves as a credible benchmark with both static and interactive evaluation modes.\n\n3. Human preference tests validate that improvements are not simply artifacts of the automatic scoring pipelines.", "weaknesses": "1. The paper claims general applicability to “code aesthetics” broadly, but without evidence from other modalities (e.g., GUI design, visualization dashboards, game UIs). A brief transfer or zero-shot study in a different domain would make the generalization claim more convincing.\n\n2. The OpenDesign benchmark depends heavily on LLM-based judges, which may lead to circularity since similar models are used for training rewards. Although the authors mention strong correlation with human judgment, the potential for overfitting to automated metrics remains.\n\n3. Because both training rewards and evaluation rely on similar LLMs, the model might learn to exploit their aesthetic preferences rather than aligning with human aesthetic standards. The human–human agreement rate leaves some uncertainty regarding the real-world validity of “aesthetic improvement.” \n\n4. The paper does not analyze how the multi-agent reward system handles ambiguous or partially successful outputs, such as non-executable but visually correct code, or misaligned interactivity cases. Including failure case studies or qualitative analyses would increase confidence in the robustness and interpretability of the proposed framework.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)