Glyph-ByT5-v2: A Strong Aesthetic Baseline for Accurate Multilingual
  Visual Text Rendering
Authors: name: Zeyu Liu; name: Weicong Liang; name: Yiming Zhao; name: Bohan Chen; name: Ji Li; name: Yuhui Yuan
Published: 2024-06-14 17:44:09

Abstract: Recently, Glyph-ByT5 has achieved highly accurate visual text rendering
performance in graphic design images. However, it still focuses solely on
English and performs relatively poorly in terms of visual appeal. In this work,
we address these two fundamental limitations by presenting Glyph-ByT5-v2 and
Glyph-SDXL-v2, which not only support accurate visual text rendering for 10
different languages but also achieve much better aesthetic quality. To achieve
this, we make the following contributions: (i) creating a high-quality
multilingual glyph-text and graphic design dataset consisting of more than 1
million glyph-text pairs and 10 million graphic design image-text pairs
covering nine other languages, (ii) building a multilingual visual paragraph
benchmark consisting of 1,000 prompts, with 100 for each language, to assess
multilingual visual spelling accuracy, and (iii) leveraging the latest
step-aware preference learning approach to enhance the visual aesthetic
quality. With the combination of these techniques, we deliver a powerful
customized multilingual text encoder, Glyph-ByT5-v2, and a strong aesthetic
graphic generation model, Glyph-SDXL-v2, that can support accurate spelling in
10 different languages. We perceive our work as a significant advancement,
considering that the latest DALL-E3 and Ideogram 1.0 still struggle with the
multilingual visual text rendering task.