COMPUTER GRAPHICS forum

EUROGRAPHICS 2024 / A. Bermano and E. Kalogerakis(Guest Editors)

Volume 43 (2024), Number 2

FontCLIP: A Semantic Typography Visual-Language Model for
Multilingual Font Applications
Yuki Tatsukawa1

I-Chao Shen1

Anran Qi1

Yuki Koyama2

Takeo Igarashi3

Ariel Shamir4

1 {tatsukawa-yuki537, ichaoshen, annranqi1024}@g.ecc.u-tokyo.ac.jp , The University of Tokyo, Japan
2 koyama.y@aist.go.jp, National Institute of Advanced Industrial Science and Technology (AIST), Japan
3 takeo@acm.org, The University of Tokyo, Japan
4 arik@runi.ac.il, Reichman University, Israel

Abstract
Acquiring the desired font for various design tasks can be challenging and requires professional typographic knowledge. Whileprevious font retrieval or generation works have alleviated some of these difficulties, they often lack support for multiple
languages and semantic attributes beyond the training data domains. To solve this problem, we present FontCLIP – a model thatconnects the semantic understanding of a large vision-language model with typographical knowledge. We integrate typographyspecific knowledge into the comprehensive vision-language knowledge of a pretrained CLIP model through a novel finetuning
approach. We propose to use a compound descriptive prompt that encapsulates adaptively sampled attributes from a font attributedataset focusing on Roman alphabet characters. FontCLIP’s semantic typographic latent space demonstrates two unprecedented
generalization abilities. First, FontCLIP generalizes to different languages including Chinese, Japanese, and Korean (CJK),
capturing the typographical features of fonts across different languages, even though it was only finetuned using fonts of Roman
characters. Second, FontCLIP can recognize the semantic attributes that are not presented in the training data. FontCLIP’s
dual-modality and generalization abilities enable multilingual and cross-lingual font retrieval and letter shape optimization,
reducing the burden of obtaining desired fonts.

1. Introduction
Acquiring a suitable font is a crucial step in many design workflow, especially when designing a poster or a banner with crosslingual characteristics. While previous works have facilitated fontretrieval [OLAH14, CWX∗ 19] and generation [WGL20, WL21],
they are often limited to the languages and attributes presented in
the training data. The available datasets [OLAH14, CWX∗ 19] only
include Roman fonts and their associated attributes, which does
not allow users to obtain fonts in other languages. Furthermore, thecurrent datasets have limited annotated attributes, which preventsusers from specifying their desired fonts freely.
In this paper, we tackle these deficiencies by defining a semantic latent space connecting language and visuals to typography to enable multilingual and cross-lingual font retrieval andediting tasks. We base our technique on modifying a pretrained
vision-language model trained on large-scale natural image andtext pair, and without requiring any additional data-gathering beyond what is already available. Models such as CLIP (Contrastive
Language–Image Pre-training) [RKH∗ 21], have demonstrated exceptional capabilities in learning aligned visual and language features. CLIP has exhibited remarkable zero-shot recognition capabilities, empowering a wide range of downstream visual recognition
tasks [KCG∗ 23,ZZL∗ 22,LBW∗ 22,ZLD22]. Additionally, the latent© 2024 The Authors. Computer Graphics Forum published by Eurographics - The European Association for Computer Graphics and John Wiley & Sons Ltd.
This is an open access article under the terms of the Creative Commons Attribution-NonCommercial
License, which permits use, distribution and reproduction in any medium, provided the original work
is properly cited and is not used for commercial purposes.

space of CLIP has been used for various content generation applications, including images [RDN∗ 22], abstract sketches [VPB∗ 22], 3D
avatars [HZP∗ 22], and artistic images [RBL∗ 21]. These works collectively highlight that CLIP’s latent space carries profound semantic understanding that can connect between language and visuals.
However, typography is a very specialized domain that is different from natural photographs, paintings, or sketches, which wereoriginally used to train CLIP. Hence, simply using the original CLIPmodel cannot effectively recognize visual typographic characteristics and establish meaningful connections with language representations, as illustrated in Figure 2. The primary reason for this is
the substantial domain disparity between the typographical image
and those portraying natural scenes. Moreover, the language used todescribe typographic data often diverges from the descriptions of
natural scenes represented in different styles.
We present FontCLIP, a CLIP-based model specifically designed
to learn a semantic typographic latent space that bridges language
and visual typographic attributes, enabling various typographic applications. By finetuning a pretrained CLIP model on font data withattribute scores, we enhance its zero-shot recognition capability
and enable it to generalize to the typography domain. The learned
features of FontCLIP enable prediction of multilingual visual typographic attributes with the ability to generalize to out-of-domain

Yuki Tatsukawa et al. / FontCLIP: A Semantic Typography Visual-Language Model for Multilingual Font Applications“fun, smile”+

FontCLIP

Retrieval

FontCLIP

FontCLIP

Retrieval

CLIP

FontCLIP

200500(b) Attribute: "strong"

FontCLIP(b) Cross-lingual font retrieval
Optimization(a) Attribute: "complex, itatic"

CLIP(a) Dual-modal font retrieval“thin, formal, legible”

+

CLIP

FontCLIP

1000(c) Language-driven vector font optimization

2000 step

Figure 1: FontCLIP enables the following typography-specific applications. (a) Dual-modal font retrieval: our method retrieves resultsthat preserve the style of the query font image (text with frame)
while incorporating the desired attributes. (b) Cross-lingual fontretrieval: our method retrieves results in other languages with asimilar query font image style. (c) Language-driven vector font
optimization: our method manipulates the shape of input lettersaligning with a set of desired attributes.

(c) Attribute: "warm"

CLIP

FontCLIP(d) Attribute: "thin"

Figure 2: Glyph images of Roman and Chinese letters sorted by (a)“complex, italic”, (b) “strong”, (c) “warm”, and (d) “thin” attributescores predicted by CLIP and FontCLIP. FontCLIP’s sorting aligns
more closely with human perception.

letter shapes based on desired attributes or font image examples,
thereby opening up exciting possibilities for font customization(see Figure 1).
To sum up, we make the following contributions:

attributes. Remarkably, we achieve these generalizations by using
an existing Roman character dataset without the need for collecting
any new data.
Finetuning FontCLIP is accomplished using a novel compounddescriptive prompt that encapsulates multiple attributes within asingle prompt. To determine the attributes included in the compound descriptive prompt, we adaptively sample them to cover the
distribution of attribute scores and convert continuous score into
sampled text. In our finetuning process, we use a randomly generated compound descriptive prompt and a font image with a random
augmentation transformation at each iteration, thereby significantly
expanding the original font attributes dataset [OLAH14].
We evaluated the performance of FontCLIP through quantitative
experiments that assessed the correlation between the predicted andmanually annotated attribute scores. Our experiments reveal that the
finetuned FontCLIP model, originally trained on Roman alphabet
characters with 37 attributes, exhibits unprecedented generalization
capabilities. First, FontCLIP is capable of generalizing to out-ofdomain languages, which makes it possible to use it for multilingualand cross-lingual font-related tasks. Second, it can generalize to outof-domain attributes, which means it can be used for font retrieval
and editing using language descriptions beyond the original attributeset. Lastly, by leveraging the dual-modality of CLIP, FontCLIP
allows users to obtain the desired fonts through both desired text
attributes and font image examples.
We demonstrate FontCLIP in two major applications: (1) a novel
dual-modal font retrieval interface that surpasses the traditional dropdown list interface in terms of user satisfaction and achieves similar
performance without using vector-based typographical features extracted from all Roman characters and, (2) a novel optimization
framework that utilizes FontCLIP latent space to manipulate vector• To the best of our knowledge, we present the first visual-language
model that learns a semantic typographic latent space. Throughexperiments and user studies, we validate its generalization abilities over multilingual and out-of-domain attributes.
• We present a novel approach to finetune a vision-language modelusing font data with attribute scores.
• We present a dual-modal font retrieval application based on FontCLIP that uses visual examples and language descriptions tosearch for appropriate fonts across different languages.
• We present an optimization-based method to modify the shapeof letters in vector representation to better match either a set of
language descriptions or a visual input image sample.
2. Related Work2.1. Vision-Language Representations and Applications
Traditional visual recognition models are often constrained in theirpracticality since they are trained to recognize a predetermined setof object categories. Hence, they require additional labeled datato generalize to new visual concepts and domains. However, recent advancements in large vision-language models pretrained onvast image-text pairs have demonstrated that such models can acquire rich image and object-level visual representations [RKH∗ 21,
JYX∗ 21, LZZ∗ 22]. These models are semantically rich because
the paired texts contain a broader set of visual concepts than any
pre-defined concept set. Thus, the learned representations can bedirectly used for downstream image recognition tasks such as image
classification [RKH∗ 21, JYX∗ 21], object recognition [KCG∗ 23],
image segmentation [ZZL∗ 22, LBW∗ 22, ZLD22], and text-image
retrieval [LROTG21] in zero-shot setting. Moreover, the learned
representations are applied to 3D shape classification [ZGZ∗ 22],
© 2024 The Authors.
Computer Graphics Forum published by Eurographics and John Wiley & Sons Ltd.

Yuki Tatsukawa et al. / FontCLIP: A Semantic Typography Visual-Language Model for Multilingual Font Applications

3D part segmentation [LZC∗ 23], 3D avatar and shape generation [HZP∗ 22, MBOL∗ 22, MKXBP22, GAG∗ 23, TGH∗ 22] andNeRF generation and manipulation [JMB∗ 22, WCH∗ 22] tasks. To
the best of our knowledge, we propose the first semantic typography visual-language model that connects language and visual
typographic attributes. By doing so, FontCLIP enables font retrieval
and manipulation tasks with a broader range of semantic concepts,
expanding the possibilities for font customization and exploration.
2.2. Font Retrieval and Interface
Font selection is the process of selecting fonts from a set of fonts
based on user-specified conditions across different formats. Whenthe desired fonts are presented as images, traditional visual fontrecognition approaches identify the typeface, weight, and slopeof text within them [WYJ∗ 15, CYJ∗ 14]. In the context of graphic
design, users frequently aim to find a font that complements theoverall design elements. As a result, they often rely on traditional
font selection interface, such as a long list of font names, whichis overwhelming to navigate and utilize. To address this issue,
O’Donovan et al. [OLAH14] proposed selecting fonts using semantic attributes and collected a font attribute dataset. More recent
advancements have introduced larger font attribute datasets and deeplearning-based methods to improve the accuracy and efficiency of
font retrieval [CWX∗ 19,CMA19]. However, these previous attributebased font selection methods only work on in-the-domain attributes
and scripts. In contrast, using FontCLIP latent space enables fontretrieval with out-of-domain attributes and scripts, thereby offeringenhanced flexibility and efficiency in font selection.

Selected Attribute
PromptScoreartistic✓

formal

✓

wide

80

10

80

✓

"This is a artistic, not formal, wide font."

CLIP Text
Encoder
CLIP Image
Encoder

Augmentation

Pairwisesimilarity loss

Figure 3: Overview of FontCLIP finetuning. During each finetuning iteration, we randomly select attributes based on their scoresfrom an existing font-attribute dataset and create a compound descriptive prompt for each font. Simultaneously, we generate a fontimage and apply a random augmentation transformation to enhancevariability. We finetuned the last three transformer blocks (highlighted in red) for both encoders using a pairwise similarity loss
function.

method using FontCLIP concentrates on capturing and reconstructing typographical features of each character (e.g., thin, italic, andserif). As we demonstrated in Figure 10, our method is more effective at capturing and reconstructing typographical features thanWord-As-Image. Second, our optimization method utilizes FontCLIP’s text and image encoder, which allows for dual-modal font
optimization. In contrast, Word-As-Image only operates on text input. This makes our optimization method more practical and usefulfor capturing fonts in real-world scenarios, as shown in Figure 13.
3. CLIP Preliminaries

2.3. Vector Font Generation
Example-based methods generate a complete character set of afont [SI10] or a personalized handwritten style [CLJ∗ 15, LZCX18]
from a single character. Parameterizing fonts is another methodthat allows users to create novel fonts by adjusting a set of parameters [SR98, Knu82]. Campell and Kautz [CK14] took a step further
and presented the first generative model for fonts. By exploring thelearned manifold, the model enables interpolation between existing fonts and the discovery of new fonts. Recently, various deeplearning-based methods have been proposed for synthesizing vector glyphs [LHES19, CDAT20, WL21, RGLM21]. However, thesemethods often require users to provide sample characters of the desired font, making them challenging if they lack such resources. To
address this issue, Wang et al. [WGL20] proposed Attribute2Font,
which generates glyph images solely based on user-specified attributes. However, they can only generate bitmap glyph images for
attributes and languages that are included in the training data. Thus,
to generate bitmap glyph images for new attributes or languages,
more training data is necessary. In contrast, our method can generate
vector fonts that are easily manipulable. Moreover, our method cangenerate vector fonts for attributes and languages that are not partof the training data without requiring additional training data.
Our vector font optimization method is inspired by Word-AsImage [IVH∗ 23], but with two significant differences. First, while
Word-As-Image focuses on deforming a vector letter toward a conceptual visual representation (e.g., cat or dog), our optimization© 2024 The Authors.
Computer Graphics Forum published by Eurographics and John Wiley & Sons Ltd.

CLIP [RKH∗ 21] is a vision-language model pretrained on a large
number of image-text paired data and trains both an image encoder
EI and a text encoder ET to a joint latent space. During training,
CLIP uses a contrastive loss to learn a joint embedding for the twomodalities. Specifically, for a mini-batch of image-text pairs, CLIP
maximizes for each matching image-text pair the cosine similarity of
their embeddings while minimizing the cosine similarities with allother unmatched texts/images. After training, the joint latent spaceof CLIP enables various downstream image processing and visiontasks in a zero-shot manner. For example, in image classification,
given an input image I, its image embedding (x = EI (I)) is foundusing the image encoder, and a set of text embeddings ({wi =ET (Ti )}K
i=1 ) are found using the text encoder. In particular, each wi
is derived from a prompt Ti , such as “a photo of a {class}” where
the “{class}” token is filled with the i-th class name. The prediction
probability of class y is then defined as:p(y|x) =exp(sim(x, wy )/τ)∑K
i=1 exp(sim(x, wi )/τ)(1)

where sim(·, ·) denotes cosine similarity and τ is a learned temperature parameter.
4. FontCLIP
Our goal is to learn a semantic typography latent space that canbe effectively used for various typographic applications, including
font retrieval and optimization-based font manipulation. To achieve

Yuki Tatsukawa et al. / FontCLIP: A Semantic Typography Visual-Language Model for Multilingual Font Applications

this goal, we focus on incorporating typography-specific knowledge into the existing large pretrained vision-language model CLIP.
Our approach is to finetune a pretrained CLIP model using pairsof descriptive prompts and font image as inputs derived from a
font-attribute dataset (Figure 3). In the following, we provide thetechnical details of our finetuning approach as well as the rationale
behind its design.

4.1. Finetuning - Training Data
For finetuning, we use the dataset from [OLAH14]. This dataset
consists of 200 Roman fonts, each annotated with 37 attribute scores.
The attributes in the dataset can be broadly classified into two categories. Some of these attributes are related to the shape of the fonts,
such as “serif”, “italic”, and “thin”. The other attributes describe
perceptual qualities such as “friendly”, “warm”, and “happy”. Eachfont in the dataset is assigned scores ranging from 0 and 100 for each
attribute. Among these attributes, there are some binary attributes:“capitals”, “cursive”, “display”, “italic”, “monospace” and “serif”,
meaning that they are assigned a score of either 0 or 100. To suit our
finetuning requirement, we modify the original dataset and create a
prompt-based dataset that aligns better with our objectives.
4.1.1. Compound Descriptive Prompt
Radford et al. [RKH∗ 21] introduced a hand-crafted prompt: “a photoof a “{class}” for generic objects and scenes. However, in the caseof our font dataset, each font is characterized by multiple attributes
simultaneously with continuous scores, which differs from the original classification task that the original CLIP model was trained on.
As a result, the above simple prompt is inadequate for accurately
describing each font in our dataset. To overcome this challenge, we
propose to use a compound descriptive prompt, combined with anadaptive sampling technique, to generate a more comprehensive anddescriptive prompt for each font in our dataset.
During each finetuning iteration i, we generate the compoundprompt TiF for a font F as
TiF = “This is [A]1 , [A]2 , ..., [A]N font.”,(2)

where each [A]n represents the n-th sampled attribute and N denotes
the total number of attributes sampled. Throughout the finetuning
process, we randomly set N to between 1 and 3 for each iteration.
To determine the expression for each attribute, we consider whether
its score is over or below 50. Specifically, if the score is over 50,
we use the expression [A]n = “[attribute]”. Conversely, if the scoreis below 50, we use [A]n = “not [attribute]”. For example, if thescore of attribute “happy” for font F exceeds 50, the corresponding
expression in the compound prompt is set to “happy”. Otherwise,
it is set to “not happy”. We randomly selected attributes for eachfont based on their attribute score distribution. Specifically, theprobability of attribute a being selected is computed asp(a) =∥S(a) − 50∥∑37
i=1 ∥S(ai ) − 50∥,

4.1.2. Font Image
To generate the training images for each font in the dataset, we
adopt the same approach used by O’Donovan et al. [OLAH14].
Specifically, we render an image of each font using the text “Thequick brown fox jumps over the lazy dog”, which is widely used
for displaying font samples due to its inclusion of a diverse rangeof letters [OLAH14, KdM20]. To enhance the robustness of ourfinetuned model, we apply standard data augmentation techniquessuch as rotation, cropping, and scaling to the font image (refer toFigure 3).

4.2. Finetuning - Loss Function
Our finetuning method does not apply the CLIP-style contrastive
learning [RKH∗ 21] directly. Instead, it only requires positive pairsthat include a compound descriptive prompt and a font image. The
reason for this is that negative expressions like “not [attribute]” arealready incorporated in our compound descriptive prompt. Thus,
we can finetune the pretrained model to learn effective semantic
typographic latent space solely by maximizing the cosine similarity
between the embedded vectors of the compound descriptive prompts
and those of their corresponding font images. Specifically, given apair of a font image I F and a compound descriptive prompt T F , wedefine the pairwise similarity loss function as:
LPS = −FF
1 n EI (Iq ) · ET (Tq ),∑F
n q=1 ∥EI (Iq )∥∥ET (TqF )∥(4)

where n is the number of font descriptive prompt and font image
pairs, EI and ET are the image and text encoder of the finetuned
CLIP model, respectively.

4.3. Implementation Details
Our finetuning approach is based on the pretrained ViT CLIP model,
which is publicly available on Hugging Face† . Throughout the finetuning process, we update the weights of the last three transformer
block layers in both text and image encoders for 3, 000 epochs,
while keeping the remaining weights frozen. We use the Adam optimizer [KB15] with a learning rate of 2 × 10−5 , which is halved
every 500 epochs. The resolution of the font image used in finetuning is 214 × 214. In our setting, the finetuning process took around12 hours on a machine equipped with an i7-12700K CPU with 32GB
memory and RTX3080 GPU with 10GB memory.

5. Experiments
We conducted experiments using in-domain and out-of-domain attributes to evaluate the performance of FontCLIP. Specifically, wemeasured the correlations between the attribute score predicted using the FontCLIP features and the ground truth attribute score using
the dataset from [OLAH14].

(3)

where S(a) represents the score of attribute a. We show how wegenerate a compound descriptive prompt in the top row of Figure 3.

† https://huggingface.co/sentence-transformers/
clip-ViT-B-32© 2024 The Authors.
Computer Graphics Forum published by Eurographics and John Wiley & Sons Ltd.

Yuki Tatsukawa et al. / FontCLIP: A Semantic Typography Visual-Language Model for Multilingual Font Applications

In-domain ↑Out-of-domain↑

CLIP
FontCLIP (w/o CDP)
FontCLIP

0.159
0.704
0.723

0.159
0.317
0.404

Table 1: The average correlations for in-domain attributes and outof-domain attributes of CLIP, FontCLIP trained without using compound descriptive prompts (w/o CDP), and FontCLIP trained with
CDP (ours). By using CDP, FontCLIP can better generalize to outof-domain attributes.

5.1. In-Domain Attributes
The first experiment aims to evaluate the consistency for in-domain
attributes, wherein all attributes are used during the finetuning ofeach model. We used all 200 fonts from [OLAH14], which we randomly divided into 140 fonts for training, 30 fonts for validation,and 30 fonts for testing. We finetuned FontCLIP using the 140 fonts
from the training set. For each font F in the testing dataset and foreach attribute [A], we calculated the similarity score between the
font and the attribute in the following process. First, we obtained thevisual embedding vector EI (IF ) of the font visual prompt IF associated with F. Next, from the attribute [A], we created a descriptive
prompt TA : “This is a [A] font.” and obtained the text embedding
vector ET (TA ) for this prompt. Finally, we calculated the cosine
similarity between EI (IF ) and ET (TA ). We considered a model’s
performance as the average correlation between predicted attributescores and ground truth scores for all fonts in the testing set acrossall attributes. We compared three models: FontCLIP without using
compound descriptive prompts (CDP), FontCLIP with CDP, and thebaseline CLIP model.
As shown in the first row in Table 1, all variants of FontCLIP surpasses CLIP by a substantial margin. This observation suggests that
FontCLIP has effectively learned the relationship between visual
typographic attributes and the corresponding semantic attributes
described by language, resulting in attribute ratings that are morealigned with human ratings. In addition, we provide a visualizationof the correlation between predicted similarity scores and groundtruth scores for the attributes “thin” and “playful” in Figure 4. Thisvisualization allows us to observe the alignment between the predicted attribute scores and the ground truth scores. We also include
correlation visualizations for other attributes in the supplemental
material.
5.2. Generalization to Out-of-Domain Attributes
To evaluate the generalization capability of the FontCLIP latent
space to out-of-domain attributes, meaning they are absent in the
finetune training data, we conducted a leave-one-out experiment.
During this experiment, we used all 200 fonts as training data for
finetuning the model but excluded one attribute at a time during
the finetuning process. Then, we calculated the average correlation
between the predicted similarity score and ground truth attributescores for all fonts, solely for the excluded attribute. This process
was repeated for each attribute in the dataset from [OLAH14], resulting in a total N finetuning process, where N represents the number© 2024 The Authors.
Computer Graphics Forum published by Eurographics and John Wiley & Sons Ltd.

CLIP

predicted cosine similarity

Model(a) Playful

FontCLIP(b) Boring

ground truth score

Figure 4: The visualization of the correlation between the predictedsimilarity scores from CLIP and FontCLIP, and the ground truthscores for (a) “playful” and (b) “boring” attributes.
Font A

Font B(a) Font A or Font B is more "formal"?

Reference Font

Font A

Font B(b) Font A or Font B is more similar to Reference Font?

Figure 5: (a) In the pairwise attribute prediction task, we use a classifier to determine which font has a higher attribute score between two
font options. (b) In the pairwise similarity prediction task, we use
a classifier to determine which font is more similar to a reference
font between two font options. In both tasks, the obtained results
are compared with human judgments, and the prediction accuracy is
calculated as the performance metric.

of attributes. The performance of each model was evaluated bycomputing the average correlation across all N attributes.
The results presented in the second row of Table 1 highlight that
FontCLIP (w/o CDP) already performs better than CLIP. Moreover,the inclusion of CDP noticeably enhances the correlations. Thesefindings indicate that FontCLIP can effectively generalize to outof-domain attributes with straightforward finetuning and compounddescriptive prompts. Additionally, we provide the correlation scoresof all attributes in the supplemental material.
6. Dual-Modal Multilingual Font Retrieval
O’Donovan et al. [OLAH14] introduced attribute-based andsimilarity-based interfaces that enhance traditional dropdown menus
with a list of font names. Inspired by their work, we proposeYuki Tatsukawa et al. / FontCLIP: A Semantic Typography Visual-Language Model for Multilingual Font Applications

Model

Accuracy ↑Model

Accuracy ↑CLIP
FontCLIP

51.87%
65.32%

CLIP
FontCLIP

67.68%
74.39%

Feature-based

65.73%

Feature-based

75.95%(a): Pairwise attribute prediction.

(b): Pairwise similarity prediction.

Model

In-domain accuracy

Out-of-domain accuracy

CLIP
FontCLIP

54.92%
64.14%

42.26%
64.48%

Feature-based

N/A

N/A

(c): CJK fonts pairwise attribute prediction.

Table 2: (a)(b) For both the pairwise attribute prediction task andpairwise similarity task, FontCLIP outperforms CLIP’s performanceand achieves similar performance to the best model that relies ongeometric typographical features computed from the vector-based
font file including all characters (“Feature-based”) [OLAH14], while
our FontCLIP-based method only requires a font image as input. (c)The FontCLIP latent space generalizes to out-of-domain attributes
and to multiple languages. Note that previous feature-based methods
require the vector-based font files, cannot recognize out-of-domain
attributes, and might be able to support multi-lingual capabillities
only if they had access to the multi-lingual fonts files (although thishas never been tested).

leveraging the FontCLIP latent space for dual-modal font retrieval
tasks. Besides the attribute-based interface, our interface facilitates image-based retrieval, which does not require the user toobtain the vector-based font files covering all characters, unlike
O’Donovan et al. [OLAH14]. Moreover, our interface allows forany combination of the attributes and images. In addition, the FontCLIP latent space exhibits the capability to generalize beyond Roman fonts, allowing it to support multiple language settings. In thefollowing sections, we present quantitative evaluations for attributebased, image-based, and cross-lingual font retrieval, along withqualitative evaluation of multilingual and cross-lingual font retrieval
using a combination of attribute and image inputs.

6.1. Quantitative Evaluation
To quantitatively evaluate the attribute-based and image-based fontretrieval, we follow the experiment setup outlined in [OLAH14]
and focus on in-domain attributes. For attribute-based retrieval, we
conduct the pairwise attribute prediction task, while for image-based
retrieval, we evaluate the pairwise font similarity prediction task.
In both tasks, we use the complete 200 fonts and the 31 adjectives
attributes in the dataset from [OLAH14]. Our main goal of FontCLIP
is to enable font retrieval without the need to access the original
vector-based font files. Therefore, we primarily focus on comparing
the performance of FontCLIP and CLIP because both methods
use font images as input instead of the vector-based font files. For
reference, we provide a performance of the best machine learningmodel using vector-based typographical features [OLAH14].

Pairwise Attribute Prediction The goal of the pairwise attributeprediction task is to determine which font has a higher attributescore between two font options represented as font images (Figure 5(a)). We first evaluate this task for in-domain attributes.In total,
we generated 198, 400 pairwise comparison subtasks for evaluating
this task. Each comparison was assessed by seven people, and wecomputed the accuracy through respective comparisons. As shownin Table 2(a), FontCLIP achieves better performance comparedto CLIP. This suggests that the FontCLIP feature is more distinguishable regarding different attributes. Moreover, despite taking
only a font image as input, FontCLIP achieves comparable performance to the best model that uses typographical features proposedby [OLAH14]. This result suggests that FontCLIP’s image-based
typographical features extracted only from target glyphs (i.e., notglyphs of all Roman characters) are representative and achieve similar retrieval performance as vector-based typographical features
extracted from all Roman characters.
Furthermore, we conducted quantitative evaluations to assess
FontCLIP’s generalization capabilities on out-of-domain attributesand different languages. Out-of-domain attributes cannot be handled by the methods in [OLAH14] because their models need to
be trained in an attribute-specific manner. In addition, their models
use typographical features that are specifically designed for Roman
characters. For the evaluations, we additionally collected pairwise
attribute rating data for 50 CJK fonts with three participants recruited from our university. The collected dataset contains ratingsfor 5 in-domain attributes and 3 out-of-domain attributes specificallyused for describing CJK fonts, including “traditional”, “robust”, and“Japanese style”. In Table 2(c), we show the results of the attributeprediction task performed on CJK fonts. It can be observed that
FontCLIP outperforms CLIP on both in-domain and out-of-domain
attributes in this task, especially on out-of-domain attributes.
Pairwise Similarity Prediction The pairwise similarity prediction
task involves selecting the font that is more similar to a given reference font image out of two font images (Figure 5(b)). This task
serves as a means to assess whether the distances in the FontCLIP
latent space accurately reflect the perceptual similarity between
fonts. In total, we generated 35, 387 comparisons for this task. Eachpairwise comparison was voted by 10 to 15 individuals, and we computed the accuracy through respective comparisons. For the analysis,
we excluded 52 comparisons with tie votes. As shown in Table 2(b),similar to the pairwise attribute prediction task, FontCLIP obtains
better performance compared to CLIP and achieves comparable
performance to the best model that uses vector-based typographical features extracted from all Roman characters [OLAH14]. The
results of both tasks indicate that FontCLIP’s image-based typographical features perform similarly to its vector-based counterpart
without requiring vector-based font files for all Roman characters,
thus significantly reducing the efforts of retrieving new fonts.
Cross-Lingual Pairwise Similarity Prediction Task We conduct
two pairwise similarity prediction tasks to assess the cross-lingual
retrieval ability of different methods, As shown in Figure 6, thefirst task is called “Roman-to-CJK”, which involves selecting the
CJK font that is more similar to a given Roman font. The second
task is “CJK-to-Roman”, which involves selecting the Roman font
© 2024 The Authors.
Computer Graphics Forum published by Eurographics and John Wiley & Sons Ltd.

Yuki Tatsukawa et al. / FontCLIP: A Semantic Typography Visual-Language Model for Multilingual Font Applications
Font B

Font A

Reference Font(a) Roman-to-CJK
Reference Font

Font A

Font B

Multilingual Font Retrieval First, we demonstrate FontCLIP’sunprecedented generalization capability by showing multilingual
font retrieval results using a combination of attributes and image
inputs. Our goal aligns with [KdM20], where we aim to retrieve
results that preserve the style of input font image while incorporating
the desired attributes. Specifically, given a query font image Iqueryand a set of desired attributes A = {a1 , a2 , ..., aN }, we obtain theembedding vector of the desired font edesired using the following
formulation:
edesired = EI (Iquery ) + wET (T ),(b) CJK-to-Roman

Figure 6: Cross-Lingual Pairwise Similarity Prediction. (a) For the“Roman-to-CJK” task, we use a classifier to determine which CJK
font is more similar to a reference Roman font between two CJK
font options. (b) Conversely, for the “CJK-to-Roman” task, we use
a classifier to determine which Roman font is more similar to areference CJK font between two Roman font options. In both tasks,
we compare the results obtained using the classifiers with humanjudgments, and calculate the prediction accuracy as the performance
metric.

Model

Roman-to-CJK ↑CJK-to-Roman ↑CLIP
FontCLIP

57.4%
67.2%

50.0%
62.6%

Table 3: For both cross-lingual pairwise similarity tasks, FontCLIP
performed better than CLIP, suggesting that FontCLIP’s prediction
results are closer to human rating results.

that is more similar to the query CJK font. To evaluate these twotasks quantitatively, we collected 280 fonts that were not part of
the training dataset. In total, we generated 100 pairwise comparison
subtasks for “Roman-to-CJK” and “CJK-to-Roman” tasks and recruited five participants to rate these comparisons. In Table 3, weshow the results for both “Roman-to-CJK” and “CJK-to-Roman”
tasks. We can observe that FontCLIP’s predictions are better alignedwith human ratings compared to CLIP’s predictions. This suggeststhat despite being finetuned solely on the Roman character dataset,the FontCLIP model can still learn general typographical featuresthat achieve better cross-lingual font retrieval. We also observed that
FontCLIP’s performance of “Roman-to-CJK” was better than its“CJK-to-Roman” counterpart, which is in line with our expectation
given that the FontCLIP model was trained only on the Roman
character dataset.

6.2. Qualitative Evaluation
In our qualitative evaluation, we collected 1, 169 Roman fonts and
293 CJK fonts in total. For each font, we generate its font image
using the method described in Section 4.1.2 and extract its visualfeature using FontCLIP visual encoder EI . We denote the final fontfeature databases for Roman and for CJK as ΩRoman and ΩCJK .
© 2024 The Authors.
Computer Graphics Forum published by Eurographics and John Wiley & Sons Ltd.

(5)where EI and ET is the image and text encoder of FontCLIP, T isa text prompt containing all desired attribute A, and w ∈ [0, 1] is aweight that controls the balance between preserving the original letter styles and incorporating the styles of the desired attributes. With
the embedding vector edesired , we obtain the top-K retrieved resultsby choosing the K closest fonts to edesired in the corresponding fontfeature database (ΩRoman or ΩCJK ).
In Figure 7, we compared the retrieved results obtained using
FontCLIP and CLIP in different languages, considering both indomain and out-of-domain attributes. In Figure 7, we found thatthe retrieved results using FontCLIP effectively incorporate thedesired attributes while preserving the original style. For example,
in Figure 7(a), more retrieved results are with serifs for all languages
by using the FontCLIP feature than that of CLIP; in Figure 7(b),for out-of-domain attribute “traditional”, the retrieved results by
FontClip also better align with the human perception than that of
CLIP. Besides, the CLIP latent space fails to interpret the “not”
prompt, resulting in thicker retrieved results compared to the results
obtained by the FontCLIP feature (Figure 7(a)).
Cross-Lingual Font Retrieval Next, we demonstrate the crosslingual font retrieval results using FontCLIP. Our goal is to retrieve
fonts that have a similar style to the query font image Iquery fromother languages. To begin with, we calculate the visual feature of thequery font image Iquery as EI (Iquery ). Following this, we search forthe k nearest fonts to the visual feature in the font feature dataset ofother languages. In the “Roman-to-CJK” results shown in Figure 8,
we have found that the FontCLIP feature is better at retrieving CJK
fonts that are more similar to the query Roman fonts, compared to
the CLIP feature (Figure 8(a)). Meanwhile, we observed that the“CJK-to-Roman” retrieved results of CLIP deviate excessively fromthe style of the query font image (Figure 8(b)).
7. Dual-Modal Multilingual Vector Font Optimization
In this section, we describe another application of FontCLIP: an
optimization-based method that modifies the letter shapes in vector
fonts based either on text prompts or on image inputs in multiple languages. Figure 9 shows the overview of our vector font optimization
method. Guided by the FontCLIP latent space, our optimizationbased method supports both language-driven and image-driven font
optimization in multiple languages.
The input to our method is a letter l from an existing font in vector
format. Following [IVH∗ 23], we represent l as a set of k control
points, P = {p j ∈ R2 }kj=1 , describing it’s outline. P is obtained by a
subdivision, which provides sufficient expressiveness even for letters

Yuki Tatsukawa et al. / FontCLIP: A Semantic Typography Visual-Language Model for Multilingual Font Applications
Roman

ChineseJapanese

Roman

CLIP

FontCLIP

Chinese

CLIP

FontCLIP

Japanese

CLIP

CLIP

FontCLIP
CLIP

FontCLIP
CLIP

FontCLIP

FontCLIP(b) Top-5 retrieval results for out-of-domain attributes: “traditional”(a) Top-5 retrieval results for in-domain attributes: “serif, not thin”

Figure 7: The results of dual-modal font retrieval using FontCLIP latent space and CLIP latent space. The goal of this multi-modal retrieval isto preserve the style of input font image query (text with frame) while incorporating the desired attributes. (a) We show the top-5 retrieval
results with in-domain attributes for Roman, Chinese, and Japanese characters. By using the FontCLIP feature, we can retrieve more fonts with
serif for multiple languages. (b) We show the top-5 retrieval results with out-of-domain attributes for Roman, Chinese, and Japanese characters.
CLIP

CLIP

FontCLIP

FontCLIP
CLIP

CLIP

FontCLIP

FontCLIP

CLIP

CLIP

FontCLIP

FontCLIP(a) Top-5 Roman-to-CJK retrieval results(b) Top-5 CJK-to-Roman retrieval results

Figure 8: Cross-lingual font retrieval results using FontCLIP latent space and CLIP latent space. The goal of this cross-lingual font retrieval is
to find fonts in other languages that match the style of the input font image query (text with frame). (a) We show the top-5 retrieval results for“Roman-to-CJK”. (b) We show the top-5 retrieval results for “CJK-to-Roman”.

Lacap

P=

Ltone

pk
p1
p2
pk

DiffVGP=

p1
p2"This is an italic, formal,
legible font."
FontCLIP
Text
Encoder

FontCLIP
Image
Encoder

Llanguage(a) Language-driven

FontCLIP
Image
Encoderfont image Iuser as additional input that drives the optimization (seeFigure 9).
LvisualFontCLIP
Image
Encoder(b) Visual-driven

Figure 9: An overview of our multi-modal vector font optimization.
Given an input letter l (“A” in this example) represented as a set ofoutline control points P, and either a language-driven descriptive
prompt Tuser (a), or a visual-driven reference font image Iuser (b), we
iteratively optimize the new positions of P̂ creating the optimized
letter shape l.ˆ Inspired by [IVH∗ 23], we first rasterize the deformed
letter lˆ by a differentiable rasterizer (DiffVG). To guide the optimization, we use a language loss Llanguage in (a) language-driven
optimization, or a visual loss Lvisual in (b) visual-driven optimization
to ensure lˆ aligns with desired attributes indicated by the descriptive prompt or the reference font image. Moreover, our objectivefunction includes the tone preservation loss Ltone and an ACAP
deformation loss Lacap similar to [IVH∗ 23]. Black and red dashedarrows indicate forward and backward computation, respectively.

with few control points. The output of our pipeline is the same setof control points P̂ = { p̂ j }kj=1 in different positions that represents
the outline of the manipulated letter l.ˆ The users define their goal
either by providing a text prompt Tuser of attributes or a reference

7.1. Language-Driven Font Optimization
Our specific goal here is to manipulate the original letter l into Iˆ byaligning it with desired attributes while preserving the original styles.
To preserve the original styles of l, we begin by calculating similarity
scores between its original shape R(P) and the 37 attributes. We
then select the top-M (we set M = 2) attributes with the highest
similarity scores as the attributes to be preserved. The user-specified
prompt Tuser is then combined with these M preserved attributes toform the final compound descriptive prompt Tfinal . To encourage
the manipulated letter lˆ to align with Tfinal , we define the following
function using the FontCLIP visual encoder EI and text encoder ET :Llanguage (P̂, Tfinal ) = dist(EI (R(P̂)), ET (Tfinal )),

(6)

x·y
where dist(x, y) = 1.0 − ∥x∥∥y∥denotes the cosine distance between

x and y, and R is the differentiable rasterizer [LLGRK20]. However,
we have noticed that using Llanguage alone can result in significant
deviations from the initial letter geometry. Inspired by [IVH∗ 23],
we incorporate the ACAP deformation loss and tone preservation
loss into our final objective function. The ACAP deformation loss
minimizes the deviation of the final letter shape from its initial shape,
while the tone preservation loss aims to preserve the font’s style andletter structure. For detailed definitions of both losses, please refer
to [IVH∗ 23]. The objective function is defined as
LLD = Llanguage + wacap Lacap + wtone Ltone ,(7)© 2024 The Authors.
Computer Graphics Forum published by Eurographics and John Wiley & Sons Ltd.

Yuki Tatsukawa et al. / FontCLIP: A Semantic Typography Visual-Language Model for Multilingual Font Applications
Optimization steps

Optimization steps“italic, formal, legible font”“thin font”“serif font”“not serif font”

WordAs-Image
FontCLIP

WordAs-Image
FontCLIP

Figure 10: Visualization of the vector font optimization steps of the language-driven Roman and Chinese character optimization using
FontCLIP. We compared the results obtained by Word-As-Image [IVH∗ 23] and our method. Our method better captures and reconstructs eachcharacter’s typographical features, including features such as serif.

Letter(a) Input font(b) SDS + T�inal(c) Our +Tuser(d) Our + T�inalT�inal : " This is an italic, formal, legible font".

Figure 11: Ablation study on the language-driven font optimization.
Given (a) an input font, we compare the results obtained by (b)replacing Llanguage into SDS loss, (c) our method using only Tuser ,and (d) our method using Tfinal . (The user specificed attributes areshown in blue and the attributes to be preserved are shown in red.)

Reference
font image

Optimization steps

30

100

500

1000

1500

2000

30

100

500

1000

3000

5000

step

step

Figure 12: Visualization of the optimization steps of the crosslingual image-driven Roman and Chinese character optimization.

7.2. Image-Driven Font Optimization

where we set wacap = 0.2 and wtone = 0.2 throughout all examples
shown in this paper. In Figure 1(b), we can observe the iterative
optimization steps where the Chinese character gradually becomesthinner while maintaining its formal and legible appearance. In Figure 10, we compare the results obtained by our method and Word-AsImage [IVH∗ 23] on Roman and Chinese characters. We can observethat our optimization method using FontCLIP feature captures andreconstructs typographical features better, even for features such asserif.

Ablation Study In Figure 11, we present a comparison of various
formulations. This includes replacing the language loss Llanguagewith the Stable Diffusion (SDS) loss, which was used in [IVH∗ 23],and solely using Tuser in Llanguage (i.e., excluding the attributes we
aim to preserve). As can be seen, the result using SDS loss did
not exhibit the desired attributes used in the text prompt Tfinal andseverely deviated from the original letter shape. The result using
Tuser reflects the desired attribute (“italic”) but fails to preserve theoriginal styles of the input font.
© 2024 The Authors.
Computer Graphics Forum published by Eurographics and John Wiley & Sons Ltd.

When a reference font image Iuser is given, image-driven font optimization manipulates l into lˆ while ensuring that lˆ reflects the visual
typographic attributes present in Iuser . To achieve this, we define thefollowing function using the FontCLIP visual encoder EI and text
encoder ET :Limage (P̂, Iuser ) = dist(EI (R(P̂)), EI (Iuser )),(8)

and the overall objective function for image-driven font manipulation is defined as:
LV D = Limage + wacap Lacap + wtone Ltone ,(9)

where we set wacap = 0.2 and wtone = 0.2. In Figure 12, we present
cross-lingual optimization results on both Roman and CJK characters. We can observe the iteratiove optimization steps that graduallyalign the styles of the input Roman and Japanese letters with thestyle in the reference font images even from other languages. Finally,in Figure 13, we demonstrate the effectiveness of our font optimization method using a reference font image captured in real-world
conditions. We extracted the letters from the captured image andused them as Iuser to drive the optimization for the provided letters.

Yuki Tatsukawa et al. / FontCLIP: A Semantic Typography Visual-Language Model for Multilingual Font Applications

9. Conclusion(a) captured reference font image(b) extracted letters(c) letter(d) optimized letter

Figure 13: (a) Given a reference font image captured in real-world,
our optimization method uses (b) the extracted letters to manipulate(c) the input letters. (d) The optimized letters exhibits a similar styleto the fonts in the captured image.

In this paper, we introduced FontCLIP – a model that bridges thesemantic understanding of a large vision-language model with typographical knowledge. Our experiments demonstrated FontCLIP’s
two unprecedented generalization abilities. First, FontCLIP cangeneralize to multiple languages despite being finetuned only ona Roman character dataset. This ability enables multilingual andcross-lingual font retrieval and letter shape optimization. Second,
FontCLIP can recognize out-of-domain semantic attributes, facilitating more diverse attribute-based font retrieval and letter shape optimization. Finally, FontCLIP’s dual-modality allows unprecedented
multilingual font applications through a unified space without extracting typographical features through vector-based font files. In
summary, we believe FontCLIP can greatly simplify the process ofobtaining desired fonts during the design process.
Acknowledgement(a) Input font(b) Optimized result(c) Input font(d) Optimized result

Figure 14: Our optimization method faces challenges in handlingcomplex structures such as crossing and rounded strokes.

8. Limitations and Future Work
Attribute Entanglement Currently, FontCLIP latent space exhibits
entanglement between different attributes. As shown in Figure 11(c),the optimized letter exhibits characteristics from attributes that are
not specificed by the user. As a result, our method need to identify
and preserve the most representative attributes of the font during
language-driven font optimization (Figure 11(d)). In the future, a potential solution would be to explore contrastive finetuning, utilizing
fonts with similar attribute scores but differing in only one attribute.
Vector Font Optimization on Complex Typographic Structures
While our current character shape optimization method shows
promising results, it faces challenges in handling complex typographic structures, such as crossing and rounded strokes (Figure 14).
Future research could improve our optimization method by investigating more appropriate font parameterizations [HHH10] and incorporating more typographic-specific constraints. Nonetheless, ourresults validate the concept and suggest that our FontCLIP could bea foundation of future font optimization methods.
Generalization Enhancement Currently, FontCLIP is specificallyfinetuned using a dataset that exclusively contains Roman alphabet
characters and commonly associated attributes. However, there is apossibility that cultural differences might affect how letter shapes
are linked to attributes. To alleviate this issue, we plan to explore
few-shot learning techniques for out-of-domain languages, which
involve collecting small-scale datasets using the data collection
process described in [OLAH14].

We thank the anonymous reviewers for their valuable feedback.
This work was partially supported by JST AdCORP, Grant Number JPMJKB2302, JSPS Grant-in-Aid JP23K16921, Japan, and acollaboration with Dentsu Digital.
References[CDAT20] C ARLIER A., DANELLJAN M., A LAHI A., T IMOFTE R.:
DeepSVG: A hierarchical generative network for vector graphics animation. In Proc. NeurIPS (2020), vol. 33, pp. 16351–16361. URL: https://proceedings.neurips.cc/paper_files/paper/2020/
file/bcf9d6bd14a2095866ce8c950b702341-Paper.pdf.
3[CK14] C AMPBELL N. D. F., K AUTZ J.: Learning a manifold offonts. ACM Trans. Graph. 33, 4 (2014). doi:10.1145/2601097.
2601212. 3[CLJ∗ 15] C HEN H.-I., L IN T.-J., J IAN X.-F., S HEN I.-C., C HEN B.-Y.:
Data-driven handwriting synthesis in a conjoined manner. Comput. Graph.
Forum 34, 7 (2015), 235–244. doi:10.1111/cgf.12762. 3[CMA19] C HOI S., M ATSUMURA S., A IZAWA K.: Assist users’ interactions in font search with unexpected but useful concepts generated by multimodal learning. In Proc. ICMR (2019), pp. 235–243.
doi:10.1145/3323873.3325037. 3[CWX∗ 19] C HEN T., WANG Z., X U N., J IN H., L UO J.: Large-scale
tag-based font retrieval with generative feature learning. In Proc. ICCV(2019), pp. 9116–9125. doi:10.1109/ICCV.2019.00921. 1, 3
[CYJ∗ 14] C HEN G., YANG J., J IN H., B RANDT J., S HECHTMAN E.,
AGARWALA A., H AN T. X.: Large-scale visual font recognition. In Proc.
CVPR (2014), pp. 3598–3605. doi:10.1109/CVPR.2014.460. 3[GAG∗ 23] G AO W., A IGERMAN N., G ROUEIX T., K IM V., H ANOCKA
R.: TextDeformer: Geometry manipulation using text guidance. In Proc.
SIGGRAPH (2023). doi:10.1145/3588432.3591552. 3[HHH10] H ASSAN T., H U C., H ERSCH R. D.: Next generation typeface
representations: Revisiting parametric fonts. In Proc. DocEng (2010),
pp. 181–184. doi:10.1145/1860559.1860596. 10[HZP∗ 22] H ONG F., Z HANG M., PAN L., C AI Z., YANG L., L IU
Z.: AvatarCLIP: Zero-shot text-driven generation and animation of 3D
avatars. ACM Trans. Graph. 41, 4 (2022). doi:10.1145/3528223.
3530094. 1, 3
[IVH∗ 23] I LUZ S., V INKER Y., H ERTZ A., B ERIO D., C OHEN -O R D.,
S HAMIR A.: Word-as-image for semantic typography. ACM Trans. Graph.
42, 4 (2023). doi:10.1145/3592123. 3, 7, 8, 9

© 2024 The Authors.
Computer Graphics Forum published by Eurographics and John Wiley & Sons Ltd.

Yuki Tatsukawa et al. / FontCLIP: A Semantic Typography Visual-Language Model for Multilingual Font Applications[JMB∗ 22] JAIN A., M ILDENHALL B., BARRON J. T., A BBEEL P.,
P OOLE B.: Zero-shot text-guided object generation with dream fields.
In Proc. CVPR (2022), pp. 867–876. doi:10.1109/CVPR52688.
2022.00094. 3[RGLM21] R EDDY P., G HARBI M., L UKAC M., M ITRA N. J.:
Im2Vec:synthesizing vector graphics without vector supervision. In Proc.
CVPR (2021), pp. 7342–7351. doi:10.1109/CVPRW53098.2021.
00241. 3[JYX∗ 21] J IA C., YANG Y., X IA Y., C HEN Y.-T., PAREKH Z., P HAM
H., L E Q., S UNG Y.-H., L I Z., D UERIG T.: Scaling up visual andvision-language representation learning with noisy text supervision. In
Proc. ICML (2021), pp. 4904–4916. URL: http://proceedings.
mlr.press/v139/jia21b/jia21b.pdf. 2[RKH∗ 21] R ADFORD A., K IM J. W., H ALLACY C., R AMESH A.,
G OH G., AGARWAL S., S ASTRY G., A SKELL A., M ISHKIN
P., C LARK J., ET AL .:Learning transferable visual models from natural language supervision.
In Proc. ICML (2021),
pp. 8748–8763. URL: https://proceedings.mlr.press/v139/radford21a/radford21a.pdf. 1, 2, 3, 4

[KB15] K INGMA D. P., BA J.: Adam: A method for stochastic optimization. In Proc. ICLR (2015). URL: https://arxiv.org/abs/
1412.6980. 4[KCG∗ 23] K UO W., C UI Y., G U X., P IERGIOVANNI A., A NGELOVA A.:
F-vlm:open-vocabulary object detection upon frozen vision and language
models. In Proc. ICLR (2023). URL: https://openreview.net/
pdf?id=MIMwy4kh9lf. 1, 2
[KdM20] K ULAHCIOGLU T., DE M ELO G.: Fonts like this but happier:A new way to discover fonts. In Proc. MM (2020), pp. 2973–2981.
doi:10.1145/3394171.3413534. 4, 7
[Knu82] K NUTH D. E.: The concept of a meta-font. Visible language 16,
1 (1982), 3–27. 3[LBW∗ 22] L UO H., BAO J., W U Y., H E X., L I T.: SegCLIP: Patch aggregation with learnable centers for open-vocabulary semantic segmentation,
2022. arXiv:2211.14813. 1, 2
[LHES19] L OPES R. G., H A D., E CK D., S HLENS J.: A learned representation for scalable vector graphics. In Proc. ICCV (2019), pp. 7930–7939.
doi:10.1109/ICCV.2019.00802. 3[LLGRK20] L I T.-M., L UKÁČ M., G HARBI M., R AGAN -K ELLEY J.:
Differentiable vector graphics rasterization for editing and learning. ACM
Trans. Graph. 39, 6 (2020). doi:10.1145/3414685.3417871. 8[LROTG21] L IU Z., RODRIGUEZ -O PAZO C., T ENEY D., G OULD S.:
Image retrieval on real-life images with pre-trained vision-and-language
models. In Proc. ICCV (2021), pp. 2125–2134. doi:10.1109/
ICCV48922.2021.00213. 2[LZC∗ 23] L IU M., Z HU Y., C AI H., H AN S., L ING Z., P ORIKLI F., S U
H.: PartSLIP: Low-shot part segmentation for 3d point clouds via pretrained image-language models. In Proc. CVPR (June 2023), pp. 21736–
21746. doi:10.1109/CVPR52729.2023.02082. 3[LZCX18] L IAN Z., Z HAO B., C HEN X., X IAO J.: EasyFont: A stylelearning-based system to easily build your large-scale handwriting fonts.
ACM Trans. Graph. 38, 1 (2018). doi:10.1145/3213767. 3[LZZ∗ 22] L I L. H., Z HANG P., Z HANG H., YANG J., L I C., Z HONG
Y., WANG L., Y UAN L., Z HANG L., H WANG J.-N., ET AL .: Groundedlanguage-image pre-training. In Proc. CVPR (2022), pp. 10965–10975.
doi:10.1109/CVPR52688.2022.01069. 2[MBOL∗ 22] M ICHEL O., BAR -O N R., L IU R., B ENAIM S., H ANOCKA
R.: Text2mesh: Text-driven neural stylization for meshes. In Proc.
CVPR (2022), pp. 13492–13502. doi:10.1109/CVPR52688.2022.
01313. 3[MKXBP22] M OHAMMAD K HALID N., X IE T., B ELILOVSKY E., P OPA
T.: CLIP-Mesh: Generating textured meshes from text using pretrained
image-text models. In Proc. SIGGRAPH Asia (2022). doi:10.1145/
3550469.3555392. 3[OLAH14] O’D ONOVAN P., L UNDEFINEDBEKS J., AGARWALA A.,
H ERTZMANN A.: Exploratory font selection using crowdsourced attributes. ACM Trans. Graph. 33, 4 (2014). doi:10.1145/2601097.
2601110. 1, 2, 3, 4, 5, 6, 10[RBL∗ 21] ROMBACH R., B LATTMANN A., L ORENZ D., E SSER P., O M MER B.: High-resolution image synthesis with latent diffusion models,
2021. arXiv:2112.10752. 1[RDN∗ 22] R AMESH A., D HARIWAL P., N ICHOL A., C HU C., C HEN M.:
Hierarchical text-conditional image generation with clip latents, 2022.
arXiv:2204.06125. 1© 2024 The Authors.
Computer Graphics Forum published by Eurographics and John Wiley & Sons Ltd.

[SI10] S UVEERANONT R., I GARASHI T.: Example-based automatic fontgeneration. In Proc. Smart Graphics (2010), pp. 127–138. doi:10.
5555/1894345.1894361. 3[SR98] S HAMIR A., R APPOPORT A.: Feature-based design of fonts
using constraints. In Electronic Publishing, Artistic Imaging, and Digital
Typography (Berlin, Heidelberg, 1998), Hersch R. D., André J., Brown
H., (Eds.), Springer Berlin Heidelberg, pp. 93–108. doi:10.1007/
BFb0053265. 3[TGH∗ 22] T EVET G., G ORDON B., H ERTZ A., B ERMANO A. H.,
C OHEN -O R D.: MotionCLIP: Exposing human motion generation toclip space. In Proc. ECCV (2022), pp. 358–374. doi:10.1007/
978-3-031-20047-2_21. 3[VPB∗ 22] V INKER Y., PAJOUHESHGAR E., B O J. Y., BACHMANN R. C.,B ERMANO A. H., C OHEN -O R D., Z AMIR A., S HAMIR A.: CLIPasso:
Semantically-aware object sketching. ACM Trans. Graph. 41, 4 (2022).
doi:10.1145/3528223.3530068. 1[WCH∗ 22] WANG C., C HAI M., H E M., C HEN D., L IAO J.: Clip-NeRF:
Text-and-image driven manipulation of neural radiance fields. In Proc.
CVPR (2022), pp. 3835–3844. doi:10.1109/CVPR52688.2022.
00381. 3[WGL20] WANG Y., G AO Y., L IAN Z.: Attribute2Font: Creating fonts
you want from attributes. ACM Trans. Graph. 39, 4 (2020). doi:10.
1145/3386569.3392456. 1, 3
[WL21] WANG Y., L IAN Z.: DeepVecFont: Synthesizing high-quality
vector fonts via dual-modality learning. ACM Trans. Graph. 40, 6 (dec2021). doi:10.1145/3478513.3480488. 1, 3
[WYJ∗ 15] WANG Z., YANG J., J IN H., S HECHTMAN E., AGARWALA
A., B RANDT J., H UANG T. S.: DeepFont: Identify your font from animage. In Proc. ICMR (2015), pp. 451–459. doi:10.1145/2733373.
2806219. 3[ZGZ∗ 22] Z HANG R., G UO Z., Z HANG W., L I K., M IAO X., C UI B.,
Q IAO Y., G AO P., L I H.: PointCLIP: Point cloud understanding by CLIP.
In Proc. CVPR (2022), pp. 8552–8562. doi:10.1109/CVPR52688.
2022.00836. 2[ZLD22] Z HOU C., L OY C. C., DAI B.: Extract free dense labelsfrom CLIP. In Proc. ECCV (2022), pp. 696–712. doi:10.1007/
978-3-031-19815-1_40. 1, 2
[ZZL∗ 22] Z HOU Z., Z HANG B., L EI Y., L IU L., L IU Y.: ZegCLIP: Towards adapting clip for zero-shot semantic segmentation, 2022. arXiv:
2212.03588. 1, 2