Font Generation with Variational Autoencoders
By: Yigit Atay
Tags: Fonts, Machine Learning, Variational Autoencoder, Deep Learning

Font is more important than we think. Font design has become one of the most crucial aspects of marketing for many mainstream consumer brands, as it represents their personality and tone. As people started to pay more and more attention to font design, especially for making logos, the market for fonts began to grow significantly. Currently, there are hundreds of font design and font database websites. So, why not make this process more customizable? Why not generate your own fonts according to your own design preferences? This is exactly what this project aims to achieve: Automate the font design process. Since this project heavily utilizes some data science tools and particularly artificial neural networks, so basic familiarity with the topics is helpful.

Collecting and Preprocessing the Data

When training a neural network, it’s important to have as much data as possible. The data for this project was collected (using Python’s requests module) from multiple font providers, namely Google, Urban Fonts, and 1001 Fonts, and there are currently about 60000 data points in the dataset. The two default file formats for font data are ttf and otf files. However, we cannot use these as input to a neural network, therefore we should first convert them to png files. For now, I’ve decided to use only alphabetical characters (uppercase and lowercase), and avoid numerical characters and symbols, because there are numerous inconsistencies in the dataset, with a good number of fonts not having numerical characters or symbols at all. After using the ttf/otf files to create 512x512 png’s, here’s what a data point looks like:

Figure 1: Example input data

Dataset Creation

The Magic of Variational Autoencoders

Variational autoencoders, or VAE’s, are pretty amazing. An autoencoder consists of two parts: an encoder that takes the data and compresses it into a much smaller representation, and a decoder that decompresses this representation to attempt to recreate the input. The difference between an autoencoder and VAE is that the compressed form of the data the encoder outputs (aka the “bottleneck”) is represented as a probability distribution of some parameters. This change allows the decoder to “sample” from the bottleneck, making VAE’s a generative model. The encoder of the VAE that is used in this project has the following structure:

Encoder model of the font generator

The latent_mu and latent_sigma variables are essentially the mean and the standard deviation of the probability distribution (or what’s called the “latent space”), and lambda is sampled from this distribution. Furthermore, a VAE’s loss function has an additional component called the KL divergence, which ensures that all encodings are distributed close to the center of the latent space, and not clustered far from each other. The advantage of having this kind of a latent space is that the model can learn very specific variations in the data. Here’s an example to understand this better: Let’s say we are training a VAE on images of faces. Where a regular autoencoder might associate a single parameter in the latent space with whether or not the the eyes are closed, a VAE can actually learn how closed the eyes are (here, the openness of the eyes becomes a continuous variable, instead of a single number).

This very useful attribute of VAE’s can be used to represent the different qualities of a font as well. Optimally, the VAE would learn to distinguish between features such as font weight, width, corner rounding, italics, contrast and serifs. Later, when we’re generating fonts, the weights of these features can be manually altered to generate a customized font.

During training, the sampled output from the encoder then goes into the decoder, which aims to reconstruct the initial input. The structure of the decoder is given below:

Decoder model of the font generator

Loading the decoder model for the font generator

Training settings for the VAE model

After training the model, the reconstruction of the font in figure 1 looks like this:

Reconstructed font from figure 1

Of course, the purpose of this model is not to reconstruct already existing fonts. After training the entire model, we can actually use the decoder with inputs of our choosing. The decoder here generates a font from this customized latent space.

The size of the latent space in the architecture that I used is 100, which means that to generate a font, we need 100 values. It is possible to generate a font using 100 random numbers in some range. However, using completely random numbers isn’t really helpful when we’re trying to understand what happens when we change a single variable. For this purpose, we can use the pyqt5 module to create a GUI with sliders for each parameter of the latent space, and the output being displayed next to them. The GUI, along with a generated font, looks like this:

Example view of the font generator GUI

Setting the ranges for the parameters in the GUI

The range of the parameters can be changed from the script (the link to the Github repository is at the bottom of this post). When the latent space is this large, it might be difficult to understand how changing a parameter affects the outcome. However, I am going to present one example of a very well-represented feature of this model. The 34th parameter seems to decide the italics of a given font.