StrokeNUWA: Tokenizing Strokes for Vector Graphic Synthesis
Authors: name: Zecheng Tang; name: Chenfei Wu; name: Zekai Zhang; name: Mingheng Ni; name: Shengming Yin; name: Yu Liu; name: Zhengyuan Yang; name: Lijuan Wang; name: Zicheng Liu; name: Juntao Li; name: Nan Duan
Published: 2024-01-30 15:20:26

Abstract: To leverage LLMs for visual synthesis, traditional methods convert raster
image information into discrete grid tokens through specialized visual modules,
while disrupting the model's ability to capture the true semantic
representation of visual scenes. This paper posits that an alternative
representation of images, vector graphics, can effectively surmount this
limitation by enabling a more natural and semantically coherent segmentation of
the image information. Thus, we introduce StrokeNUWA, a pioneering work
exploring a better visual representation ''stroke tokens'' on vector graphics,
which is inherently visual semantics rich, naturally compatible with LLMs, and
highly compressed. Equipped with stroke tokens, StrokeNUWA can significantly
surpass traditional LLM-based and optimization-based methods across various
metrics in the vector graphic generation task. Besides, StrokeNUWA achieves up
to a 94x speedup in inference over the speed of prior methods with an
exceptional SVG code compression ratio of 6.9%.