# Identifying Synthetic Faces through GAN Inversion and Biometric Traits Analysis
## 3 Inversion-Based Detection
### Feature Extraction and Classification Process  
After the inversion and reconstruction, we aim at jointly characterizing the visual appearance of \(\mathbf{x}_{\text{I}}\) and \(\mathbf{x}_{\text{R}}\), with the goal of predicting whether the former has been synthesized through \(G\) or not. Thus, in all cases, the objects of our analysis will be _images face pairs_, which we indicate as being _real_ and _synthetic_ when the related \(\mathbf{x}_{\text{I}}\) is real or synthetic, respectively. In particular, as depicted in Figure 4, we extract a feature representation separately from each image in the pair, thus obtaining \(\mathbf{F}_{\text{I}}\) and \(\mathbf{F}_{\text{R}}\). Then, a comparison operator between the two is applied, thus obtaining for each pair a single _differential feature vector_. Those are then used to train a classifier, to automatically characterize the differences resulting from the inversion process.  
We employ different feature extractors and comparison rules with the goal of capturing specific biometric traits, resulting in different types of differential feature vectors. In particular, we adopt as feature extractors both deep embeddings and handcrafted features proposed in the literature for automated face analysis tasks, namely:  
* _FaceNet embeddings_: Proposed in [26], the FaceNet features are the best-performing ones on the LFW face recognition dataset [27] among the deep features selected in the Deepface toolbox ([[https://github.com/serengil/deepface](https://github.com/serengil/deepface)]([https://github.com/serengil/deepface](https://github.com/serengil/deepface)) (accessed on 30 December 2022)). In computing \(\mathbf{F}_{\text{I}}\) and \(\mathbf{F}_{\text{R}}\), we employ the original 512-dimensional FaceNet version and its compact 128-dimensional variant. In this case, the comparison is simply an element-wise difference in the module. We denote as \(FN_{128}\)\(\in\mathbb{R}^{128}\) and \(FN_{512}\)\(\in\mathbb{R}^{512}\) the two types of differential feature vectors obtained.
* _Facial landmarks_: proposed in [28] and available in the [[https://github.com/davisking/dlib](https://github.com/davisking/dlib)]([https://github.com/davisking/dlib](https://github.com/davisking/dlib)) (accessed on 30 December 2022) library, the landmark localization algorithm returns 68 facial landmarks related to key facial structures. Those can be further partitioned into different face areas (face line, eyebrows, eyes, nose, and mouth), as is shown in Figure 5. This feature extractor outputs the arrays \(\mathbf{F}_{\text{I}}\) and \(\mathbf{F}_{\text{R}}\) of size \(68\times 2\), containing row-wise, the 2D coordinates of the 68 landmarks, and we extract from them two types of differential feature vectors:
* \(LM_{68}\)\(\in\mathbb{R}^{68}\) contains the Euclidean distances between \(\mathbf{F}_{\text{I}}[i,:]\) and \(\mathbf{F}_{\text{R}}[i,:]\), \(i=1,\ldots 68\) (i.e., the 2D coordinates of corresponding landmarks in the two different faces);
* \(LM_{136}\)\(\in\mathbb{R}^{136}\) contains the differences in module between individual corresponding landmark coordinates \(\mathbf{F}_{\text{I}}[i,j]\) and \(\mathbf{F}_{\text{R}}[i,j]\), \(i=1,\ldots 68\), \(j=1,2\).  
Figure 4: Pipeline of the comparison and training/classification processes.