# Evaluation of StyleGAN-CLIP Models in Text-to-Image Generation of Faces
## 1 Introduction  
Generative models [1, 2, 3] are machine learning models that aim to generate novel and realistic data on the basis of the probability distribution, which describes a given training set. Such models can find applications in many fields such as generating realistically looking human faces [4], animating art works [5], composing music [6] and augmenting datasets with synthetic data [7]. However, the usefulness of these models depends on both their efficiency in creating images and the output quality. Two successful model architectures for synthesising images are the Generative Adversarial Network (GAN) [8, 9, 10] and the diffusion model [11, 12]. The former requires less computational resources whilst the latter is known to produce higher quality images [13, 14]. In this study, we limit ourselves to GANs since we are interested in fast computation.  
The GAN model is made up of two models that compete against each other to learn to synthesise realistic data. The two modules, the generator and the discriminator, are two independent neural networks. The generator synthesises fake data starting from random noise input, while the discriminator, with the help of a dataset of real images, classifies whether the input data is from the dataset or from the generator. These two modules are trained one at a time. The generator is trained to produce outputs that the discriminator categorizes as real, while the discriminator is trained to correctly classify the new fake data. After several iterations, the generator is expected to generate data that looks real even to a human, whilst the discriminator is unable to distinguish real from fake data.  
Different network topologies have been used in defining the GAN architecture to implement the generator and discriminator modules, including Deep Convolutional GAN (DCGAN) [15], Conditional GAN (CGAN) [16], InfoGAN [17] and StyleGAN [18], the latter considered the state-of-the-art GAN in generating non-conditioned images [19]. In addition to their ability to generate high-quality face images, StyleGAN models provide the opportunity to control the style of the generated photo by altering the style vector inputs. The model was improved with its successors StyleGAN2 [19] and StyleGAN3 [20]. Nevertheless, the default StyleGAN architecture generates images unconditionally from simple latent code features rather than from some controllable input such as a text prompt, which restricts its applications. Since text can conveniently describe what a user requires, it can be explored to control the latent space directly to achieve text-guided image generation and editing tasks. Recently, a number of works [21; 22; 23] have explored the use of Contrastive Language Image Pretraining (CLIP), a multimodal embedding space model [24], to manipulate the StyleGAN model input vector in order to control the text-to-image output. We leverage upon these works to set up the experimental system for our study.  
In this paper, we compare the output quality of five StyleGAN models combined with CLIP to generate images of faces from textual descriptions. These models are StyleGAN [18], StyleGAN2 [19], StyleGAN2-adaptive (StyleGAN2-ADA) [25], StyleGAN2-Distillation (StyleGAN2-Dist) [26] and StyleGAN3 [20]. In addition, following the works described in [21; 27], we study the image editing task. The comparative analysis is carried out on the basis of automatic metrics, as well as human evaluations, and in addition, we obtain some insight into the correlation of automatic evaluation metrics with human ratings.  
The rest of this paper is organised as follows. In Sections 2 and 3, the StyleGAN and CLIP models used in this work and the text-to-image evaluation metrics from the literature are reviewed. In Section 4, the experimental methodology is described, and in Section 5, the models are evaluated and the results are discussed. Section 6 concludes the paper and proposes future work.