## 4 Experiments and Analysis
### Main Experiment  
In order to verify the effectiveness of the proposed method, we execute some adopted methods, including the DeOldify method based on deep learning [37], the CycleGAN method using UNet as the generator [30], the CycleGAN method using ResNet as the generator [31], and the S-Net method based on UNet [6]. The corresponding results are plotted in Figure 7.  
As shown in Figure 7, satisfactory results are obtained. First, our method is better at coloring vegetation and making it closer to the actual image in the scene, while the DeOldify and CycleGAN_UNet methods did not estimate the correct luminance information, resulting in vegetation in the NIR image being \"transparent\". Compared with CycleGAN_UNet, the color of vegetation and sky of CycleGAN_ResNet is improved well, but there are some color errors, such as the street signs and roads in the seventh row are painted blue and the mud in the eighth row is painted green. Second, compared to results from the S-Net method, our method contains more detailed texture information, such as the sky in the sixth row and trees and buildings in ninth row have clearer textures. Third, the building generated is with a reasonable color rather than being gray, like those obtained by DeOldify, or being uneven, like those obtained by CycleGAN_UNet, CycleGAN_ResNet, and S-Net. Finally, although the color of our reconstructed image cannot be completely consistent with that of the original image, our method can obtain an RGB image with more natural appearance. In the examples of images in the fifth and tenth rows, the ground truth itself has a less obvious color, but the color of the tree becomes clearer and brighter after coloring by our method.  
For quantitative evaluations, three evaluation metrics are used, including PSNR (Peak Signal to Noise Ratio) [38], MAE (Mean Absolute Error), and SSIM (structural similarity index) [39].  
PSNR is an evaluation index to measure image quality. The higher the value, the better the image quality. It is defined by mean square error (MSE).  
\[PSNR=10\times log_{10}\Bigg{(}\frac{\left(2^{n}-1\right)^{2}}{MSE}\Bigg{)} \tag{10}\]  
Figure 7: Visual comparison of different methods: (**a**) input NIR images; (**b**) DeOldify via deep learning [37]; (**c**) CycleGAN_UNet [30]; (**d**) CycleGAN_ResNet [31]; (**e**) S-Net [6]; (**f**) our method; and (**g**) corresponding RGB images.  
where n is the bit depth of image and MSE is represented by the following formula:  
\[MSE=\frac{1}{H\times W}\sum_{i=1}^{H}\sum_{j=1}^{W}\left(Y(i,j)-Y^{\prime}(i,j) \right)^{2} \tag{11}\]  
where \(H\) and \(W\) represent the height and width of images, respectively, \(Y\) represents the ground truth, \(Y^{\prime}\) represents the predicted RGB image, and (_i,j_) represents the pixel position.  
MAE is used to calculate the pixel-level mean absolute error between the predicted RGB image and the ground truth and the formula can be expressed as:  
\[MAE=\frac{1}{n}\sum_{i}^{n}\left|Y_{i}-Y^{\prime}_{i}\right| \tag{12}\]  
where \(n\) is the number of pixels.  
SSIM measures image similarity from three aspects: brightness, contrast, and structure, respectively. The \(SSIM\) is defined as:  
\[SSIM=l\left(Y,Y^{\prime}\right)\times c\left(Y,Y^{\prime}\right)\times s\left( Y,Y^{\prime}\right) \tag{13}\]  
where  
\[\begin{array}{l}l\left(Y,Y^{\prime}\right)=\frac{2\mu_{Y}\mu  
Furthermore, our method also performs the best on both S-CIELAB and LPIPS metrics. In summary, the proposed method achieving NIR colorization is obviously superior to the existing methods.