# Identifying Synthetic Faces through GAN Inversion and Biometric Traits Analysis
## 3 Inversion-Based Detection
### Inversion Process  
Let us consider the typical GAN architecture, where a generator \(G\) and a discriminator \(D\) are trained jointly through an adversarial process. The goal of \(G\) is to generate synthetic data that resemble real data; the goal of \(D\) is to correctly distinguish the synthetic data generated by \(G\) and an available corpus of real data [25]. Starting from an initial version of both, the two networks are trained in an alternate manner by competing with each other and progressively improving their performance: the current version of \(D\) is fine-tuned on the real samples, and the synthetic ones created by the current version of \(G\); in turn, \(G\) is then fine-tuned so as to maximize the classification loss of the updated version of \(D\). At the end of this training process, the distribution of data generated by \(G\) is intended to match the distribution of real data, so as to minimize the discrimination capabilities of \(D\). Figure 1: Overview of the inversion-based detection. Generators typically work to transform a randomly drawn vector \(\mathbf{z}\in\mathcal{Z}\) into an image \(\mathbf{x}=G(\mathbf{z})\) in the image space \(\mathcal{X}\). \(\mathbf{z}\) and \(\mathcal{Z}\) represent the _latent vector_ and the _latent space_, respectively, and \(\mathbf{z}\) encodes information about the appearance of \(G(\mathbf{z})\). Figure 2 depicts this mechanism. In other words, during the training process, the mapping \(G:\mathcal{Z}\rightarrow\mathcal{X}\) is learned, and points that are close in the latent space \(\mathcal{Z}\) are transformed through \(G\) into visually similar images in \(\mathcal{X}\). The _inversion_ (or _projection_) of GANs [12] consists of mapping a certain image under analysis \(\mathbf{x}_{\text{I}}\) back into its corresponding representation in the latent space. Formally, this can be formulated as finding a point \(\mathbf{z}_{\text{R}}\) in \(\mathcal{Z}\) such that \(G(\mathbf{z}_{\text{R}})\) is as close as possible to \(\mathbf{x}_{\text{I}}\) according to a certain metrics \(\ell(\cdot,\cdot)\):  
\[\mathbf{z}_{\text{R}}=\arg\min_{\mathbf{z}\in\mathcal{Z}}\ell(G(\mathbf{z}), \mathbf{x}_{\text{I}}) \tag{1}\]  
We denote \(\mathbf{x}_{\text{R}}\doteq G(\mathbf{z}_{\text{R}})\) as the _reconstructed_ version of \(\mathbf{x}_{\text{I}}\). In this work, we study the inversion and reconstruction processes by focusing on the case where the generator \(G\) is the widely known StyleGAN2 face generator [3], for which the authors also provide a strategy for solving the problem in (1) (see Section 5 of [3]), together with an open source implementation ([[https://github.com/NVlabs/stylegan2-ada-pytorch](https://github.com/NVlabs/stylegan2-ada-pytorch)]([https://github.com/NVlabs/stylegan2-ada-pytorch](https://github.com/NVlabs/stylegan2-ada-pytorch)) (accessed on 30 December 2022)). Figure 3 reports examples of StyleGAN2 images and their reconstructions. Therefore, given a face image under analysis \(\mathbf{x}_{\text{I}}\), possibly coming from a variety of sources, we propose to compare it to its reconstructed version \(\mathbf{x}_{\text{R}}\). If \(\mathbf{x}_{\text{I}}\) has been actually  
Figure 3: Examples of face images before (top row) and after (bottom row) the reconstruction using the inversion StyleGAN2 process available [[https://github.com/NVlabs/stylegan2-ada-pytorch/blob/main/projector.py](https://github.com/NVlabs/stylegan2-ada-pytorch/blob/main/projector.py)]([https://github.com/NVlabs/stylegan2-ada-pytorch/blob/main/projector.py](https://github.com/NVlabs/stylegan2-ada-pytorch/blob/main/projector.py)) (accessed on 30 December 2022). Figure 2: GAN architecture and training scheme. synthesized by the considered generator \(G\), its reconstruction \(\mathbf{x}_{\text{R}}\) should coincide or be very close to \(\mathbf{x}_{\text{I}}\), also depending on the effectiveness of the optimization strategy employed to solve the problem in (1).