Code Aesthetics with Agentic Reward Feedback
Authors: name: Bang Xiao; name: Lingjie Jiang; name: Shaohan Huang; name: Tengchao Lv; name: Yupan Huang; name: Xun Wu; name: Lei Cui; name: Furu Wei
Published: 2025-10-27 12:32:33

Abstract: Large Language Models (LLMs) have become valuable assistants for developers
in code-related tasks. While LLMs excel at traditional programming tasks such
as code generation and bug fixing, they struggle with visually-oriented coding
tasks, often producing suboptimal aesthetics. In this paper, we introduce a new
pipeline to enhance the aesthetic quality of LLM-generated code. We first
construct AesCode-358K, a large-scale instruction-tuning dataset focused on
code aesthetics. Next, we propose agentic reward feedback, a multi-agent system
that evaluates executability, static aesthetics, and interactive aesthetics.
Building on this, we develop GRPO-AR, which integrates these signals into the
GRPO algorithm for joint optimization of functionality and code aesthetics.
Finally, we develop OpenDesign, a benchmark for assessing code aesthetics.
Experimental results show that combining supervised fine-tuning on AesCode-358K
with reinforcement learning using agentic reward feedback significantly
improves performance on OpenDesign and also enhances results on existing
benchmarks such as PandasPlotBench. Notably, our AesCoder-4B surpasses GPT-4o
and GPT-4.1, and achieves performance comparable to large open-source models
with 480B-685B parameters, underscoring the effectiveness of our approach.