W ord A rt Designer: User-Driven Artistic Typography Synthesis using Large Language Models (2023)

Abstract: This paper introduces WordArt Designer, a user-driven framework for artistic typography synthesis, relying on the Large Language Model (LLM). The system incorporates four key modules: the LLM Engine, SemTypo, StyTypo, and TexTypo modules. 1) The LLM Engine, empowered by the LLM (e.g. GPT-3.5), interprets user inputs and generates actionable prompts for the other modules, thereby transforming abstract concepts into tangible designs. 2) The SemTypo module optimizes font designs using semantic concepts, striking a balance between artistic transformation and readability. 3) Building on the semantic layout provided by the SemTypo module, the StyTypo module creates smooth, refined images. 4) The TexTypo module further enhances the design’s aesthetics through texture rendering, enabling the generation of inventive textured fonts. Notably, WordArt Designer highlights the fusion of generative AI with artistic typography. Experience its capabilities on ModelScope: https://www.modelscope.cn/studios/WordArt/WordArt.

## introduction typography, a critical intersection of language and design, finds extensive applications across various domains like advertising @xcite @xcite , early childhood education @xcite , and historical tourism @xcite . despite its widespread relevance, the mastery of typography design remains an intricate task for non-professional designers. although attempts have been made to bridge this gap between amateur designers and typography @xcite , existing solutions mainly generate semantically coherent and visually pleasing typography within predefined concepts. these solutions often lack adaptability, creativity, and computational efficiency. to overcome these limitations, we introduce wor-dart designer (fig. 1 ), a system composed of four primary modules: the llm engine, semtypo module, and stytypo module, supplemented by the textypo module for texture rendering. this userfocused system allows users to define their design needs, including design concepts and domains. the system consists of: ## wordart designer the wordart designer system utilizes an assortment of typography synthesis modules, propelled by a large language model (llm) such as gpt-3.5), facilitating an interactive, user-centered design process. as illustrated in fig. 3 , users define their design needs, including design concepts and domains, e.g., "a cat in jewelry design." the llm engine interprets the input, generating prompts to guide semtypo, stytypo, and textypo modules, thus executing the user's design vision. to achieve automated wordart design, we introduce a quality assessment feedback mechanism, which is vital for successful synthesis. the output from the ranking model is evaluated by the llm engine to validate the quality of the synthesized image, ensuring the creation of at least k quali-fied transformations. if this criterion is not met, the llm engine, along with the semtypo and sty-typo modules and format directives, are restarted for another design iteration. subsequent sections will delve into the details of each module's functionality and operation. ## deployment details wordart designer tool has successfully been integrated into modelscope, utilizing the tongyi qian-wen 14b model as the llm engine. in terms of deployment, the stytypo and textypo modules are hosted in docker containers, ensuring both flexibility and scalability in deployment. stytypo is powered by a linux platform with 48 cores, 384gb ram, and 4 nvidia v100 32gb gpus, taking roughly 32 seconds to generate 4 images. in contrast, textypo operates within a similar linux environment but with 24 cores, 64gb ram, and a single nvidia v100 32gb gpu, and typically produces 4 images within a span of 5 to 10 seconds. for the ranking model, the mmpretrain (contributors, 2023) is used to train a resnet 18 model @xcite , with a total of 100 epoches at a batch size of 32. the sgd optimizer was used with a learning rate of 0.01. the training ran on a single nvidia v100 16gb gpu. ## experiments creativity & artistic ability. we operationalize the concept of texture rendering to evaluate the creativity and artistic ability of the wordart designer. the outcomes are demonstrated in fig. 7 . the first row of art words is generated by embodying the concept "food", which is further specified to "candy", "pasta", "cheese", "fruits", "bread", "vegetables" or "chocolate". the second row represents the concept "jewelry", concretized to "jewels", "gold" or "jade". the smart and reasonable texture rendering contributes to the creativity and artistic appeal of the output. expandability to different languages. our sem-typo module, grounded on differentiable rasterization, is theoretically compatible with all types of languages. beyond chinese (i.e., hieroglyphs), we explore the expandability of wordart designer with the representative language, english (i.e., the latin alphabet). fig. 8 presents a collection of rendered results for chinese characters and corresponding english words, substantiating that wor-dart designer effectively accommodates different languages. ## ethical considerations potential ethical concerns include perpetuating cultural stereotypes due to the use of certain imagery or symbols in the process of artistic transformations, or introducing bias against under-represented cultures. another issue could be the potential inclusion of copyrighted graphics. users need to pay attention to these issues to ensure responsible and respectful use of the system. ## conclusion this paper presents wordart designer, a framework that harnesses large language models (llm), such as gpt-3.5, to automatically generate multilingual artistic typography. this system uses an llm engine to parse and translates user input into directives, guiding three modules, each accountable for different aspects of the typographic design. the superior performance of wordart designer highlights the potential of ai to augment artistic typography. future work aims to further explore the possibilities of integrating this technology into other aspects of design, such as graphics and interactive media.