# Thermal Image Generation for Robust Face Recognition
## 4 Thermal Face Generation System
### Results of the Proposed System  
Regarding the proposed system, the StyleCLIP model cannot be used directly with thermal images because the CLIP algorithm was trained with visible images and text. On the other hand, the GNR model adds thermal styles randomly, so it is necessary to use classifiers to select the images we want in our database. Thus, to create our proposal it is necessary to train the GNR model with visible and thermal images and train the thermal classifiers. The image generation model, GNR, was trained with the PUCV-VTF database, using 15,668 visible and thermal images. To evaluate the result obtained by GNR, two metrics, _FID_[51] and LPIPS [52], are used. Frechet Inception Distance (_FID_) is a metric of the differences in the density of two distributions in the high-dimensional feature space of an InceptionV3 classifier, comparing activations of a previously trained classification network on real and generated images. To calculate the _FID_, Equation (4) is used:  
\[FID=\left\|m-m_{w}\right\|_{2}^{2}+Tr\Big{(}\ C+C_{w}-2\ (CC_{w})^{1/2}\Big{)} \tag{4}\]  
The Wasserstein-2 distance is combined with the low-dimensional embedded vector from the network Inception V3. The \(m\) and \(C\) parameters are the mean vectors and covariance matrices in the embedding space. The subscript \(w\) corresponds to the generated image, while the terms without subscripts indicate the real image. A low _FID_ value shows a better generation of synthetic images. Learned Perceptual Image Patch Similarity, LPIPS is a metric that calculates the perceptual similarity between two images using deep network activations. A small LPIPS score indicates mode collapse and lack of diversity when varying style codes. In our case, we generated ten output images for each test image, from which we calculated their LPIPS distance. Then, we averaged all distances for all test images. To compare the performance of our thermal generation imaging system, we have selected the Pix2 pix [53] model. Pix2 pix is a conditional GAN designed for image-to-image translation; therefore, thermal images are generated from visible images (as our proposed system does). The Pix2 pix model is characterized by a U-Net type generator [54] and a PatchGAN type discriminator [55]. We used the same database (PUCV-VTF) as the GNR model to train this model. In addition, we used 100 epochs and the parameters of the original Pix2 pix article [53] for the training process. This way, Table 1 presents the results obtained for the GNR model and Pix2 pix. Note that the _FID_ is lower for the GNR model while for Pix2 pix, the _FID_ is higher; this implies that the translation from visible images to thermal images is inadequate for the Pix2 pix model. The LPIPS value is not so low for both models, which indicates that the generator has diversity when generating thermal images. We trained the thermal classifiers to select six attributes (fromn, glasses, normal, rotation, smile, and vocal) to generate the synthetic database. We used 1644 images for each class for training these models, obtained from the PUCV-VTF database. Table 2 shows the result of classifying a test set, which contains 1567 thermal images never seen before by the models. We can highlight the InceptionResnetV2 method with a performance of 0.9827 of accuracy. This classifier will be used to build our automatic thermal imaging system. Finally, we conducted a thermal imaging generation experiment using the proposed model. As a result of the iterative process described in Figure 4, a database composed of 103,137 thermal images was obtained, corresponding to 3327 different subjects, with 31 images for each person in the synthetic database. The processing time used to generate the thermal database was 7 h and 30 min. Figure 7 presents the results of some images generated with the proposed system. Note that the results obtained are diverse; they show different subjects with the six selected attributes.