# Visible and Near-Infrared Image Synthesis Using PCA Fusion of Multiscale Layers
## 1 Introduction  
Recent improvements in image sensors have enabled cameras to realize better quality pictures. Many types of image-quality enhancement algorithms, such as high dynamic range (HDR), are also available today. In the field of ultrasmall cameras embedded in smartphones, cameras can be built with various functions by miniaturizing and improving performance. Cameras can now acquire both infrared and visible (VIS) images, and this capability is utilized in various ways, for example, for unlocking the smartphone function or capturing good-quality photos using the infrared camera. Infrared cameras are often used to capture a significant amount of information that lies beyond the VIS spectrum. The near-infrared (NIR) spectrum covers the wavelength range of 700-1100 nm. Therefore, infrared light is less scattered than VIS light, and hence, infrared images can capture more information, such as fog and shade. Capturing infrared images is also advantageous for obtaining details and edge information that are not generally acquired in the VIS spectrum. Many types of algorithms for fusing VIS and NIR images have been developed [1; 2; 3; 4; 5]. Most of these algorithms are intended for using NIR scenes to reveal hidden areas in VIS scenes. The conventional VIS and infrared images are used for dehazing. To synthesize VIS and NIR images, most existing methods use multiresolution decomposition and multiresolution-based convergence algorithms. Multiresolution is essential for obtaining details of NIR images. Infrared scenes can preserve image dehazing, high contrast, and edge information in dark areas better. Laplacian-Gaussian pyramids and the local entropy-fusion algorithm are conventional VIS-NIR fusion algorithms [1]. Local entropy is utilized by synthetic algorithms to transmit the maximum information from VIS and NIR images. Conventional methods also use the multiresolution decomposition of the Laplacian-Gaussian pyramid, wherein the decomposed multiple resolution images are gently blended in each layer using a local-entropy synthesis method. In short, this conventional algorithm uses a Laplacian-Gaussian pyramid for multiresolution fusion andgenerates a weighting map using the local entropy to control local contrast and visibility to generate the fusion result. However, the details of the NIR image are not well preserved, and the color expression is not natural. VIS and infrared image-fusion algorithm is a latent low-rank representation fusion algorithm (i.e., low-rank fusion) [5]. Low-rank fusion can be decomposed into low-rank parts (i.e., a global structure) and saliency parts (i.e., a local structure). The weighted-average strategy is used to fuse the low-rank parts to preserve edge information. Then, the saliency parts are fused using the sum strategy, and the final fusion image is obtained from the sum of the low-rank and saliency parts. The latent low-rank fusion algorithm decomposes an input VIS and an infrared image into a low-rank part and a saliency part of each other. After decomposition, each low-rank and saliency part of the VIS and infrared images are fused together by the sum strategy. Finally, the resulting image is obtained using the fused low-rank and saliency parts. However, this algorithm fails to capture the details of the NIR image and does not properly indicate the edge information during color compensation. In this study, image fusion of VIS and NIR images using the Laplacian pyramid and principal component analysis (PCA) is proposed. Fusion with NIR images is adopted for enhancing image details. Hence, an image is separated by a Laplacian pyramid for details and a base layer in VIS and NIR image is used to enhance the sharpness of the image. The proposed method is presented to create a weighting map for a particular detail area and to skip the PCA calculation for unnecessary areas. The values of the weighting map are obtained properly from the PCA results of the VIS and NIR scenes to improve image details.