# Multi-Sensor Face Registration Based on Global and Local Structures
## 1 Introduction  
Image fusion can often analyze and extract the complementary information of multi-sensor data, and compose a robust or informative image, which can provide more complex and detailed target scene representation [1; 2; 3]. Since the fusion process of multi-sensor images is in accordance with human visual perception, it has an important research significance for object detection and target recognition in the areas of remote sensing image, medical imaging analysis, military target detection and video surveillance [4; 5; 6]. E.g., face recognition is one of the major applications of multi-sensor data, such as visible and thermal infrared images. Visible images can provide face texture details with high spatial resolution, while thermal infrared images are not likely to be interfered with by illumination variation or face disguise with high thermal contrast. Therefore, it is beneficial for face recognition to fuse the multi-sensor data, which can combine the advantages of texture detail and thermal radiation information in the multi-spectral face images. However, image registration is a prerequisite for the success of multi-spectral image fusion, which is an essential and challenging step in the process of image fusion research [7]. Currently, multi-spectralimages registration can generally be implemented in two ways: Hardware-based registration and software-based registration. In order to ensure that the multi-spectral images are strictly geometrically aligned, hardware-based registration can be realized based on the coaxial catadioptric optical system through the beam splitter [8]. But the sophisticated and cumbersome imaging sensor equipment for generating co-registered image pairs may not be practical in many face recognition scenarios, due to its high cost and low availability. In contrast, software-based registration can capture dual band images simultaneously and independently by utilizing different off-the-shelf low cost wavebands sensors. Since no additional hardware is needed, it may be more appropriate for practical detection and recognition scenarios compared to the hardware-based registration [9]. In general, software-based registration can be classified into two categories: Area-based and feature-based approaches [10]. Area-based methods attempt to deal directly with the image intensity values to detect, for instance, cross-correlation [11], Fourier transform and mutual information [12; 13]. On the contrary, feature-based methods attempt to indirectly extract salient structures or features in the images, such as points of high curvature, corners, strong edges, intersections of lines, structural contours and silhouettes within the images [14; 15; 16]. For visible and thermal infrared image pairs usually involve quite different intensity values, and the area-based methods are good usually only on a selected small region of the images. Instead, salient structures with a strong edge could often represent a significant common feature of the heterogeneous images. In view of this, this paper focuses on the feature-based registration methods for the visible and thermal infrared face images. Feature-based registration methods first extract different feature point sets of salient structures in the multi-spectral images, and then the registration problem can be simplified to determine the correct correspondence and the inherent spatial transformation between two point sets of extracted features. Unlike the face recognition problem [17; 18], the multi-sensor face registration problem has not enough common features available to build a convolutional neural network (CNNs) framework to predict the classification model; a popular strategy is to regard the alignment of two point sets as probability density estimation, e.g., the Gaussian Mixture Model (GMM) [19; 20; 21] and Student's t Mixture Model (SMM) [22; 23; 24].