# Boosting Face Recognition under Drastic Views Using a Pose AutoAugment Manner
## 3 Proposed Methods
### Face Augmentation  
To increase intra-class pose variations for each subject, we employ a 3D graphic engine to generate multi-views from a 3D face. Both types of the 3D face (reconstructed 3D face and face scan) are compatible with our face augmentation method. In this paper, we use face scans due to their accurate 3D shape and simplicity of texture mapping to a 3D shape via camera calibration. The face augmentation pipeline for face scans consists of four components: preprocessing, pose autoaugment, face cropping, and random background appending.  
Preprocessing is necessary to obtain a smoothed face scan since a raw face scan contains lots of noise, irregular edges, and hollows. Irregular edges and hollows mainly appear in the eyes, hair, and ears, due to their reflective properties and self-occlusion. Surface noise is substantially affected by the precision of scanners. To remove noise, irregular edges, and hollows, preprocessing involves cropping the facial region, filling hollows, and denoising. First, the position of the nose tip is located. The points in which the distance from the nose tip exceeds a threshold are removed. Then, we use the bilinear interpolation algorithm to fill missing points. Lastly, we employ the Laplacian algorithm to obtain a smoothed face scan.  
In terms of pose autoaugment, we simulate a camera in a 3D context to project a face scan to 2D faces with a 3D graphic engine. Millions of views from a single 3D face can be generated by exploring the extrinsic parameters of the camera: azimuth \(R_{a}\), elevation \(R_{e}\), and in-plane \(R_{i}\) rotation. Although a pose-invariant face recognition model performs better with more views, it may induce a heavy load if we use too many views in the following tasks, such as cropping, background adding, and training. To keep a balance between the variety of poses and the resource consumption, we generate a repository of enough poses with searched distribution, and randomly sample poses from the repository for each subject.  
To be specific, a number \(N\) and distribution parameters \(W_{P}\) are set to generate enough extrinsic parameters \(\{R_{a},R_{e},R_{i}\}\). Then a group of parameters for each subject is randomly sampled from these \(\{R_{a},R_{e},R_{i}\}\). Given \(N\) and \(W_{P}\), \(W_{X}\) is first generated obeying the uniform distribution of \((0,W_{total})\), where \(W_{P}=\{w_{1},w_{2},...w_{i},...,w_{m}\}\), \(W_{X}=\{W_{x_{1}},W_{x_{2}},...,W_{x_{N}}\}\) and \(W_{total}=\sum_{i=1}^{m}w_{i}\). \(X\) is an integer vector \(\{x_{1},x_{2},...,x_{j},...,x_{n}\}\), and \(x_{j}\) is obtained by computing  
\[W_{x_{j}}\leq\sum_{i=1}^{x_{j}}w_{i}\quad\text{and}\quad W_{x_{j}}\geq\sum_{i =1}^{x_{j}-1}w_{i}. \tag{1}\]  
To those rotation parameters \(R\) for a subject,  
\[R=sX+B, \tag{2}\]  
where \(s\) is a scale value. \(B=\{b_{1},b_{2},...,b_{n}\}\), obeys the uniform distribution of \((-\theta,\theta)\). \(\theta\) and \(s\) are set manually.  
By changing \(W_{P}\) and \(N\), we can obtain the distribution and as many rotation parameters as we want. To improve the recognition performance on profile faces automatically, we tune the distribution \(W_{P}\) and \(N\) to find the proper number of profiles and the near-frontal faces using Bayesian Optimization [38].  
We crop the generated faces by reserving the same aspect ratio but do not align them for the following two reasons. First, as suggested by the literature on VGG-face training [39], face recognition achieves better performance when training faces are not aligned. Furthermore, when the pose is larger than 60 degrees, the aligned face will be seriously distorted as a result of the occluded eye corner and mouth corner.  
The background of each generated face is transparent, leading to high contrast on the facial contour. To prevent the CNN classifier from overfitting unrealistic contour patterns, we synthesize thebackground in a flexible manner by randomly appending a scene image as the background. The alpha channel is employed to combine the generated face and background.