# SADG: Self-Aligned Dual NIR-VIS Generation for Heterogeneous Face Recognition
## 4 Experiment
### Ablation Study  
In this section, we verify the effectiveness of the four segments that were used in our proposed self-aligned generation architecture, including the same z, self-aligned block, slight-weight reconstruction loss and multiscale patch discriminator. Concretely, we set the four contrast experiments as a, b, c and d. Experiment a used a different z code for training in our method, b used the same z code but without our self-aligned block, c used a normal weighted reconstruction loss when training the network and d trained the network without a multiscale patch discriminator.  
As for experiment a, when using two different z codes for training, the generated NIR-VIS images showed slight misalignment. As exhibited in Figure 8a, the NIR image of the second person had a misaligned eyebrow with that of the VIS image. In addition, the third paired NIR-VIS images had slightly different poses with each other. The MLD68 in experiment a increased to 4.7 (lower than 6.2 in DVG and higher than 3.1 in ours), and the Rank1 accuracy decreased to 97.2% (lower than 99.9% in ours). This clearly verifies the fact that different z codes can affect the alignment performance virtually, and it can help to bring better alignment and recognition performance with the same latent code z.  
As tabulated in Table 3, b gained a lower MLD68 (3.5) and better recognition accuracy (99.2%) than a, which demonstrates a more important role that the same z value played in the alignment task. That is to say, the same z affected the alignment performance virtually, while the self-aligned block acted the part for redressing the unaligned attributes subsidiarily.  
As for experiment c, we can find the worst results in both appearance and performance in Figure 8c and Table 3. The MLD68 of c reached 5.8, which was nearly close to that of DVG (6.2). Images in Figure 8c show diverse attributes such as the pose, expression and eyebrows. We owe the poor results to the normal weighted reconstruction loss that could close the pix-level distance between the raw data and the reconstructed data by a strong constraint in the training stage. Thus, the results were unaligned, as was the raw data.  
When removing the multiscale patch discriminator in experiment d, the generated images were quite bleary, as shown in Figure 8d, and the MLD68 of d increased to 4.1 by 1 pix value per keypoint due to the challenge of detection with blurry images. However,  
Figure 8: Generated results for the ablation study with contrast experiments (**a–d**). (**a**) The same z code. (**b**) Without the self-aligned block. (**c**) Normal weighted reconstruction loss. (**d**) Without a multiscale patch discriminator.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & **Same z** & \begin{tabular}{c} **Self-Aligned** \\ **Block** \\ \end{tabular} & \
\begin{tabular}{c} **Low Weighted** \\ **Reconstruction Loss** \\ \end{tabular} &
\begin{tabular}{c} **Multiscale Patch** \\ **Discriminator** \\ \end{tabular} & **MLD68** & **Rank1 (\%)** \\ \hline Baseline & \(\times\) & \(\times\) & \(\times\) & \(\times\) & 6.2 & 99.9 \\ a & \(\times\) & \(\surd\) & \(\surd\) & \(\surd\) & 4.7 & 97.2 \\ b & \(\surd\) & \(\times\) & \(\surd\) & \(\surd\) & 3.5 & 99.2 \\ c & \(\surd\) & \(\surd\) & \(\times\) & \(\surd\) & 5.8 & 96.5 \\ d & \(\surd\) & \(\surd\) & \(\surd\) & \(\times\) & 4.1 & 99.4 \\ SADG & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) & 3.1 & 99.9 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Results of the ablation study. a: the same z value; b: without the self-aligned block; c: normal weighted reconstruction loss; d: without a multiscale patch discriminator.  
blurry images slightly affected the recognition performance, because the heterogeneous recognition network was insensitive toward the quality of the images, which was different from human beings. The Rank1 accuracy of d only decreased to 99.4% by 0.5%, as shown in Table 3.