# Multi-Sensor Face Registration Based on Global and Local Structures
## 3 The Proposed Fusion Strategies for Visible and Infrared Face Images  
After obtaining the accurate alignment of the visible and infrared images, the second problem is then to solve the multi-spectral images fusion. In order to make sure that the fused image can preserve both the apparent details of the visible image and thermal radiation information of the infrared image, we propose the fusion strategies based on Guided Filtering and Gradient preserving (GF-GP). Firstly, we use two-scale image decomposition and reconstruction with a guided filtering algorithm to get an initial fusion image, and then the output fusion image is optimized by combining the intensity distribution of the infrared image and the intensity variation of the visible image. According to [33], since each source image can be separated into a base layer containing the large-scale details, and the detail layer containing the small-scale details, the two-scale image fusion \(\mathbf{F}_{G}\) is given by the guided filtering algorithm as follows:  
\[\mathbf{F}_{G}=\overline{\mathbf{B}}+\overline{\mathbf{D}}, \tag{15}\]where the fused base layer \(\overline{\mathbf{B}}\) and the fused detail layer \(\overline{\mathbf{D}}\) can be denoted as a weighted average cost of the base \(\mathbf{B}_{n}\) and detail \(\mathbf{B}_{n}\) layers of different source images:  
\[\overline{\mathbf{B}}=\sum_{n=1}^{N}W_{n}^{B}\mathbf{B}_{n},\ \overline{\mathbf{D}}=\sum_{n=1}^{N }W_{n}^{D}\mathbf{D}_{n}, \tag{16}\]  
where \(\mathbf{W}_{n}^{B}\) and \(\mathbf{W}_{n}^{D}\) represent the refined weight maps of the base \(\mathbf{B}_{n}\) and detail layers \(\mathbf{D}_{n}\) via guided image filtering respectively. In order to optimize the initial fusion image \(\mathbf{F}_{G}\), here the visible, infrared and fused images are defined by \(\mathbf{V},\mathbf{I}\) and \(\mathbf{F}\), respectively. Given that the thermal radiation information in the infrared image is typically characterized by the pixel intensity values, the pixel intensity values are quite different between the target and background. Thus, we constrain the fused image \(\mathbf{F}\) to have the similar pixel intensity distribution with the infrared source image \(\mathbf{I}\), and the optimization problem can be formulated as:  
\[\delta_{1}=\|\mathbf{F}-\mathbf{I}\|, \tag{17}\]  
where \(\|\bullet\|\) denotes the \(l_{1}\) norm. Besides, given that the pixel intensity distribution in the same physical location might be discrepant for infrared and visible face images, they are manifestations of two different phenomena. Note that the detailed information of object edge and texture is mainly characterized by the pixel gradient values in the visible image. Hence, we constrain the fused image \(\mathbf{F}\) to have the similar pixel gradient values with the visible source image \(\mathbf{V}\), the optimization problem can also be formulated as:  
\[\delta_{2}=\|\
abla\mathbf{F}-\
abla\mathbf{V}\|\, \tag{18}\]  
where \(\
abla\) denotes the gradient operator. By combining the Equations (17) and (18), the optimization problem of the fusion image \(\mathbf{F}\) can be formulated as minimizing the following objective function:  
\[\delta_{F}=\delta_{1}+\alpha\delta_{2}=\|\mathbf{F}-\mathbf{I}\|+\alpha\|\
abla\mathbf{F}- \
abla\mathbf{V}\|, \tag{19}\]  
where \(\alpha\) is a weighting factor, the optimal \(\mathbf{F}\) could be found by using gradient descent strategy. Furthermore, it is important to note that if the \(\alpha\) is small, the fusion image preserves the more thermal radiation information of the infrared image, otherwise, the fusion image preserves more edge and texture information of the visible image. We set the weighting coefficient \(\alpha=5\) in our experiment, because it can achieve good subjective visual quality in most cases.