## 4 Experiments
### Effectiveness of Enriching Intra-Variation  
To validate the effectiveness of our proposed method of enriching intra-variation via 3D-modeling-based image generation, we constructed twelve groups of training data as follows.  
* We randomly chose one sample per person in CelebA dataset [54] to form the original SSPP training data (denoted as _Original_). Note that face images in CelebA are in the wild images with varying PIE as well. However, their pose angles are relatively small (mostly within 30 degrees).
* The 3D modeling module was used to recover the intrinsic properties (albedo and shape) and external factors (i.e., PIE) of the face images in _Original_. We then synthesized images with the same PIE and background as the images in _Original_ to form another training set, named _Synthetic_.
* With the recovered properties of the images in _Original_, we further generated another seven face images for each person with varying poses, varying illuminations, varying expressions, varying poses and illuminations, or varying poses, illuminations, and expressions. We call the respective databases of generated images as _AugP_, _AugI_, _AugE_, _AugPI_, and _AugPIE_, respectively.
* By combining the above-generated images with the original images, we obtained another five training sets, denoted as _Ori_ + _AugP_, _Ori_ + _AugI_, _Ori_ + _AugE_, _Ori_ + _AugPI_, and _Ori_ + _AugPIE_, respectively.  
We trained the implementation of our proposed method with enriching intra-variation (see Figure 3a) by using the above training sets and evaluated its face verification performance on seven of the benchmark datasets, i.e., LFW [57], CPLFW [58], CALFW [59], AgeDB [60], CFP [61], VGGFace2-FP [62], and FERET-b [56]. During the test, the features extracted from the images by the network were used to judge whether two images are from same persons based on the cosine similarity between their features. Here, ten-fold cross validation was employed. Specifically, we divided the dataset into ten subsets, nine of which were used as the training set and the remaining one used as the test set.  
The results are reported in Table 1. By comparing the recognition rate obtained by training the network on _Original_ and that obtained by training the network on _Synthetic_, it can be found that they are comparable. This indicates that the shape and albedo obtained by using our proposed 3D modeling method can effectively capture the identity information. When the network was trained with the generated data only, the recognition accuracies on most of the datasets were improved. Moreover, the face recognition accuracy was consistently improved when training the network with the combined set of original and generated images. It is worth mentioning that our proposed method of enriching intra-variation improved the recognition accuracy by \(8.07\%\) (\(=58.82\%-50.72\%\)) and \(13.02\%\) (\(=82.69\%-69.67\%\)) on the challenging AgeDB and CFP-FF test sets, respectively. It can be found that pose factor is an important factor that affects the verification rate by comparing the results of AugP, AugI, and AugE or the results of _Ori+AugP_, _Ori+AugI_, and _Ori+AugE_. While the results of using merely the augmented data (i.e., _AugP_, _AugI_, _AugE_, _AugPI_, and _AugPIE_) show consistent improvement when more variations are taken into consideration, combining the original and augmented data of expression variations, however, obtained only marginal improvement in some of the test datasets. This is probably because the original data from the CelebA database already have some expression variations and play a major role in the training, and consequently the contribution of augmented expression data becomes marginal.