Symbolic Graphics Programming with Large Language Models
Authors: name: Yamei Chen; name: Haoquan Zhang; name: Yangyi Huang; name: Zeju Qiu; name: Kaipeng Zhang; name: Yandong Wen; name: Weiyang Liu
Published: 2025-09-05 16:10:53

Abstract: Large language models (LLMs) excel at program synthesis, yet their ability to
produce symbolic graphics programs (SGPs) that render into precise visual
content remains underexplored. We study symbolic graphics programming, where
the goal is to generate an SGP from a natural-language description. This task
also serves as a lens into how LLMs understand the visual world by prompting
them to generate images rendered from SGPs. Among various SGPs, our paper
sticks to scalable vector graphics (SVGs). We begin by examining the extent to
which LLMs can generate SGPs. To this end, we introduce SGP-GenBench, a
comprehensive benchmark covering object fidelity, scene fidelity, and
compositionality (attribute binding, spatial relations, numeracy). On
SGP-GenBench, we discover that frontier proprietary models substantially
outperform open-source models, and performance correlates well with general
coding capabilities. Motivated by this gap, we aim to improve LLMs' ability to
generate SGPs. We propose a reinforcement learning (RL) with verifiable rewards
approach, where a format-validity gate ensures renderable SVG, and a
cross-modal reward aligns text and the rendered image via strong vision
encoders (e.g., SigLIP for text-image and DINO for image-image). Applied to
Qwen-2.5-7B, our method substantially improves SVG generation quality and
semantics, achieving performance on par with frontier systems. We further
analyze training dynamics, showing that RL induces (i) finer decomposition of
objects into controllable primitives and (ii) contextual details that improve
scene coherence. Our results demonstrate that symbolic graphics programming
offers a precise and interpretable lens on cross-modal grounding.