# Synthetic Face Discrimination via Learned Image Compression
## 4 Methodology and Experimental Results
### Results  
#### 4.3.1 Performance Analysis  
In this analysis, we compare the suggested approach with one of the best available methods for identifying GAN-produced images. [PERSON] et al. [21] reviewed multiple methods in their work. We focus on the variant of ResNet50 that involves the removal of downsampling from the first layer. This is a technique that is still used by many researchers to compare their results, since it works exceptionally well for GAN-generated images and is still unbeatable in some cases [15; 25; 52; 53].  
The metrics we used for the evaluation of our binary classifier were accuracy, precision, and recall (also known as True Positive Rate--TPR), as well as the area under the curve (AUC) of the receiver operating characteristic (ROC) curve. The first three metrics are defined in Equations (3)-(5), utilizing True Positives (TP), True Negatives (TN), False Positives (FP), and False Negatives (FN). Unlike accuracy, and like cross-entropy losses, AUC evaluates all the operational points of the model using a Riemann sum. The AUC is calculated using the height of the recall values with the False Positive Rate (FPR). This is defined in Equation (6). A linearly spaced set of 200 classification thresholds is used for the computation of TPR-FPR pairs [54]. The ROC curve illustrates the trade-off between these two metrics. The left side of the curve represents the more \"confident\" thresholds--a higher threshold results in lower recall and fewer False Positives. On the other hand, the right side of the curve corresponds to the \"less strict\" thresholds--a lower threshold increases both the recall and the False Positives. The ROC AUC is calculated by measuring the area under the ROC curve and has values from 0 to 1, though this is often changed to percentage form  
\begin{table}
\begin{tabular}{l l l l l} \hline \hline
**Datasets** & **CelebA\_HQ** & **StyleGAN** & **StyleGAN2** & **Stable Diffusion** \\ \hline Class & Real & Synthetic & Synthetic & Synthetic \\ Training & 10,000 & 10,000 & 10,000 & None \\ Testing & 2000 & 2000 & 2000 & 2000 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Face datasets used for training and testing.  
for ease. In Figure 8, we provide a graphical example of obtaining the ROC curve and the AUC from a chosen experiment.  
\[accuracy=\frac{TP+TN}{TP+FP+TN+FN} \tag{3}\]  
\[precision=\frac{TP}{TP+FP} \tag{4}\]  
\[recall=\frac{TP}{TP+FN} \tag{5}\]  
\[FPR=\frac{FP}{FP+TN} \tag{6}\]  
The initial experiments focus on evaluating the overall efficacy of the techniques being tested in discriminating synthetic face images. The results are given in Tables 3 and 4.  
The accuracy consistently remains above 95% regardless of the generation method of the dataset. As we can see in Tables 3 and 4, we achieve better results with the StyleGAN dataset than StyleGAN2, for which our results are within 1% of those of ResNet50. Although for StyleGAN2 the accuracy of the detection is worse, it is still above 95%. The big difference between the results of the proposed method and ResNet50 comes when they are tested on images generated with stable diffusion. In this case, our compression-based method surpasses Resnet50 by far on all fronts. While Resnet50 achieves 49.70% accuracy, successfully detecting only real images, our technique does even better than with the GAN-generated datasets and reaches 99.00% and 99.10% accuracy. The precision and recall results are equally good, reaching almost 100%.  
Figure 8: The ROC curve represents the performance of our binary classifier on a StyleGAN synthetic image after the application of a 3 \(\times\) 3 median filter at different classification thresholds. The default threshold is 0.5, but to obtain the ROC curve we use values from 0 to 1. The calculation of the AUC metric requires the measurement of the area under the ROC curve (0.961).