# Visible and Near-Infrared Image Synthesis Using PCA Fusion of Multiscale Layers
## 4 Radiance Map Generation  
The proposed algorithm is used to generate a radiance map and to achieve color compensation. The flowchart of the proposed algorithm is shown in Figure 3, where \(L_{VIS}\) is the luminance channel of the VIS image, and \(L_{NIR}\) is the luminance channel of the NIR image. The input VIS and NIR images can be taken by a multi-image sensor device, which is a prism-based multi-band imaging device. It uses the same light path for VIS and NIR image capturing and can be synchronized between spatially separated VIS and NIR images, so it can be filmed without a time delay. Figure 2: Principal component analysis (PCA) fusion block diagram. The proposed radiance map algorithm is used in the Laplacian pyramid, and PCA is used in the CIELAB luminance channel. The Laplacian pyramid of input images is decomposed into edge images and base layers. Each layer is fused by the PCA fusion algorithm using the soft-mixing method. The proposed PCA fusion algorithm has a PCA-skipping mask filter. It allows skipping PCA calculations for unnecessary areas by transferring the pixel value of such areas to the next pixel area without calculating the PCA, as shown in Figure 4.  
The criteria to determine whether to implement the PCA is based on the pixel value in the PCA-skipping mask-filter area. If the pixel value in the area is less than 10% of the total pixel value in the histogram distribution of the input image, the PCA algorithm is not implemented. The pixel value that is not calculated using the PCA algorithm is set to 0.5 PC weight, as shown in Figure 4. The lower 10% of the histogram distribution of the entire pixel value is skipped because most edge-layer images have pixel values close to zero, except for the edge components. The proposed filter mask determines  
Figure 3: Flowchart of the proposed algorithm. whether the pixels in the mask-filter area should be calculated using the PCA fusion algorithm. The size of the PCA filter mask depends on the size of the Laplacian pyramid. The soft-mixing method fuses two images in the transition areas, as shown in Figure 5[10]. The transition area range is a linear fusion of input images between \(th_{1}\) and \(th_{2}\) areas and those between \(th_{3}\) and \(th_{4}\) areas. In Figure 5, \(th_{1}\) represents the value of the lower 10%, and \(th_{2}\) represents the value of the top 10% of the total in the histogram distribution of PCA weights, as shown on the \(x\)-axis in Figure 5a. Further, \(th_{3}\) denotes the value of the lower 10%, and \(th_{4}\) denotes the value of the top 10% of the total in the histogram distribution of the normalized VIS luminance histogram, as shown on the \(x\)-axis of Figure 5b. Equation (10) is the PCA fusion formula for the VIS and NIR edge layers adapted to Equation (9):  
\[rad_{detail}=P_{detail}\times VIS_{detail}+\ (1-P_{detail})\ \times NIR_{detail}, \tag{10}\]  
where \(P_{detail}\) is the detail weighting map of the PCA weights applied by the soft-mixing method, and \(VIS_{detail}\) and \(NIR_{detail}\) are the edge-layer images calculated from the VIS and NIR input images, respectively, with the Laplacian pyramid. Equation (11) is the fused expression of PCA in the base layer:  
\[base_{fused}=P_{base}\times VIS_{base}+(1-P_{base})\times NIR_{base}, \tag{11}\]  
where \(P_{base}\) the base weighting map of the PCA weights applied using the soft-mixing method, and \(VIS_{base}\) and \(NIR_{base}\) are the base-layer images obtained by calculating the VIS and NIR input images, respectively, using the Laplacian pyramid. Figure 4: PCA-skipping mask-fusion method. Figure 5: Soft-mixing method for each layer: (**a**) Edge-layer fusion using a PCA level and (**b**) base-layer fusion using a visible (VIS) luminance level. Equation (12) applies the fused image obtained using Equation (11) and the NIR images to enhance the local contrast and detail information in the base-layer images.