# Tampered and Computer-Generated Face Images Identification Based on Deep Learning
## 4 Experimental Results
### Imbalanced Dataset Experiment  
In this section, an experiment is conducted to evaluate the performance of three different extensions of the TFID model to deal with the IDP. They include XGBoost from the ensemble-based approach, class weight from cost-sensitive learning approach, and data-based approach. The proportion of tampered images to real images ranging from 1/1 (balanced dataset) to 1/100 (highly imbalanced dataset) is applied to the MANFA dataset. A total of 2000 tampered images and 200,000 real images are chosen from the MANFA dataset. Table 5 depicts the number of real and tampered face images for each imbalanced case. For the ensemble-based approach, the output 9216 feature vectors from the flatten layer are extracted, whereas 2048 feature vectors are extracted from the SE-ResNet-50 model. After that, XGBoost \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **Accuracy** & **Precision** & **Recall** & **AUC** \\ \hline TFID & 0.83 \(\pm\) 0.02 & 0.81 \(\pm\) 0.01 & 0.89 \(\pm\) 0.02 & 0.86 \(\pm\) 0.03 \\ VGG16 & 0.81 \(\pm\) 0.08 & 0.78 \(\pm\) 0.04 & 0.84 \(\pm\) 0.03 & 0.83 \(\pm\) 0.04 \\ SE-ResNet-50 & 0.84 \(\pm\) 0.04 & 0.82 \(\pm\) 0.03 & 0.91 \(\pm\) 0.02 & 0.89 \(\pm\) 0.05 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Performance of the proposed model and two state-of-the-art models on the MANFA dataset. Figure 8: TFID model interpretation via class activation mapping (CAM). For each case, a left image shows tampered image with highlighted manipulated region(s) (red circle), while the corresponding right image represents the left image CAM. classifier is trained based on these extracted features. The learning rate for XGBoost is set to 0.1, the number of trees to fit is 100, and the maximum tree depth for base learners is 3. For the cost-sensitive learning-based approach, every sample from the tampered class is considered as \(n\) instances of the real class. Therefore, a classifier is forced to treat the tampered class and the real class equally. This assumption is implemented by using the _class_weight_ parameter from Keras library, which assigns a higher loss to the tampered class to make the classifier focus more on samples from tampered class. The _class_weight_ for each class was set different according to the proportion of tampered images to real images. For example, when the proportion of tampered images to real images is 1/100, _class_weight_ is set 100 for the tampered class, while _class_weight_ for the real class is 1 to force the model to treat every instance of the tampered class as 100 instances of the real class. On the other hand, when the proportion of tampered images to real images is 1/1, which indicate a balanced dataset, _class_weight_ is fixed to 1 for both tampered class and real class. For the data-based approach, data augmentation transformation, including horizontal flip, horizontal and vertical shift, brightness, zooming, noise addition, random rotation within 10 degrees is implemented. After that, it is integrated into the python imbalanced-learn library to create a balanced batch generator, which ensures that the number of samples per class always follows a balanced distribution. Finally, eight models, including TFID, data-based TFID classifier (TFID-SA), cost-sensitive learning-based TFID classifier (TFID-CW), ensemble-based TFID classifier (TFID-XGB), SE-ResNet-50, cost-sensitive learning-based SE-ResNet-50 classifier (SE-ResNet-50-CW), data-based SE-ResNet-50 classifier (SE-ResNet-50-SA), and ensemble-based SE-ResNet-50 classifier (SE-ResNet-50-XGB) are implemented. The performance of each model on various imbalanced dataset settings are shown in Figure 9.  
Figure 9: Area under the curve (AUC) values of different TFID extensions on different imbalanced dataset scenarios, where the number of tampered images is minor compared to the number of real images (No. tampered / No. real = 1/1, 1/5, 1/10, 1/25, 1/50, 1/100).