Calligrapher: Freestyle Text Image Customization
Authors: name: Yue Ma, name: Qingyan Bai, name: Hao Ouyang, name: Ka Leong Cheng, name: Qiuyu Wang, name: Hongyu Liu, name: Zichen Liu, name: Haofan Wang, name: Jingye Chen, name: Yujun Shen, name: Qifeng Chen

We introduce Calligrapher, a novel diffusion-based framework that
innovatively integrates advanced text customization with artistic typography
for digital calligraphy and design applications. Addressing the challenges of
precise style control and data dependency in typographic customization, our
framework incorporates three key technical contributions. First, we develop a
self-distillation mechanism that leverages the pre-trained text-to-image
generative model itself alongside the large language model to automatically
construct a style-centric typography benchmark. Second, we introduce a
localized style injection framework via a trainable style encoder, which
comprises both Qformer and linear layers, to extract robust style features from
reference images. An in-context generation mechanism is also employed to
directly embed reference images into the denoising process, further enhancing
the refined alignment of target styles. Extensive quantitative and qualitative
evaluations across diverse fonts and design contexts confirm Calligrapher's
accurate reproduction of intricate stylistic details and precise glyph
positioning. By automating high-quality, visually consistent typography,
Calligrapher surpasses traditional models, empowering creative practitioners in
digital art, branding, and contextual typographic design.