# Infrared Image Enhancement Using Convolutional Neural Networks for Auto-Driving
## 4 Experiments  
The experiments were conducted by using the deep learning framework TensorFlow 2.8.0 on a GPU RTX 2080 Ti. Both the input and output image sizes were 200 \(\times\) 200. Before training the model, the input image was first transformed into a grayscale image, and then, normalized to input the model. [PERSON] was used as the optimizer, and the learning rate was set to 0.0001. The batchsize and epoch were set to 8 and 50, respectively.  
Let \(\{X_{i},Y_{i}\}_{i=1}^{N}\) be the training dataset, where \(X_{i}\) denotes the input infrared image, and \(Y_{i}\) denotes the corresponding output image. N is the number of training pairs. The infrared images used for training were derived from the FLIR thermal dataset, which contains a total of 14,000 8-bit images. FLIR is a thermal imaging dataset with a large number of low-contrast thermal imaging images, including mainly pedestrians and cars [24; 25]. Complex image components and low contrast make it extremely difficult to recognize targets, hence the need for contrast enhancement of infrared images.  
For the dataset of FLIR, similar images in the dataset were first removed and 6500 images were selected. Then, 500 images were randomly selected from these images to be flipped 45\({}^{\circ}\), 90\({}^{\circ}\), 135\({}^{\circ}\), 180\({}^{\circ}\), 225\({}^{\circ}\), 270\({}^{\circ}\), and 315\({}^{\circ}\) to enrich the training set and improve the quality of the model, which resulted in a total of 4000 images. The original 6000 images and the 4000 images obtained after flipping were treated as labeled images.  
Finally, the contrast of the target image was reduced to obtain the training image. A training set containing 10,000 pairs was created, and 1000 images were selected from the dataset to form the test set.  
The structural similarity index measure (SSIM) [26] and mean square error (MSE) [27] loss functions were used for the image enhancement class regression tasks. The SSIM is defined as follows:  
\[SSIM(x,y)=\frac{\left(2\mu_{x}\mu_{y}+c_{1}\right)\left(2\delta_{xy}+c_{2} \right)}{\left(\mu_{x}^{2}+\mu_{y}^{2}+c_{1}\right)\left(\delta_{x}^{2}+ \delta_{y}^{2}+c_{2}\right)} \tag{12}\]  
where \(x\) is the original image, and \(y\) is the target image. \(\mu_{x}\) and \(\mu_{y}\) are the means of \(x\) and \(y\), respectively. \(\delta_{x}^{2}\) and \(\delta_{y}^{2}\) are the variances of \(x\) and \(y\). \(\delta_{xy}\) is the covariances of \(x\) and \(y\), respectively. \(c_{1}\) and \(c_{2}\) are constant to maintain stability, and are defined as follows:  
\[c_{1}=(k_{1}L)^{2} \tag{13}\]  
\[c_{2}=(k_{2}L)^{2} \tag{14}\]  
where \(L\) is the dynamic range of the image, \(k_{1}=0.01\), and \(k_{2}=0.03\). The MSE is defined as  
\[MSE(x,y)=\frac{1}{N}\sum_{i=1}^{N}(x_{i}-y_{i})^{2} \tag{15}\]  
where \(x_{i}\) and \(y_{i}\) are the pixel points of \(x\) and \(y\), respectively.  
Figure 2 illustrates the loss changes with iteration number for MSE and SSIM. After two epochs, the loss values of the two types are reduced to \(1/2\) of that in the first epoch. After four epochs, the change in MSE loss is very small, but the change in SSIM loss is larger. Therefore, in terms of convergence speed, the MSE loss can reach stability in fewer epochs during training.  
Figure 2: MSE and SSIM loss functions.