# Visible Spectrum and Infra-Red Image Matching: A New Method
## 3 Proposed Method
### Regression Modeling Using Corresponding Descriptors
The corresponding descriptors  
\[T_{1} = [t_{1_{1}},\ t_{1_{2}},\ t_{1_{3}},\ \cdots,\ t_{1_{m}}], \tag{10}\]4 Feature points are shown as ‘+’ and the repeated /corresponding feature points as ‘o’. are obtained from each image pair of the training set and are appended as rows to from matrices \(R\) and \(T\). SIFT detector gives on average 380 corresponding feature points per image pair with projection error of equal to less than 2 pixels. If there are 10 image pairs in the training set, then total number of corresponding descriptors (i.e rows of \(R\) and \(T\)) obtained is \(380\times 10=3800\) and the unknown \(\Theta\)s of the model functions are learnt on these corresponding descriptors as explained above. In fact \(\Theta\)s represent the parameters of a regression model. In this paper we use five different regression models to compute the proposed \(Reg\)-SIFT. These regression models are Linear Regression (LR), Decision Tree Regression (DTR), Random Forest Regression (RFR), Support Vector Machine Regression (SVMR) and Multi-Layer Perceptron Regression (MLPR). All these models are implemented in Python with Sklearn library except the MLPR, which is implemented using the Keras and Tensorflow libraries. Figure 1 shows \(Reg\)-SIFT block, which is obtained by processing the MN-SIFT descriptors of test VS images with the trained regression model. We are using five different models; therefore, \(Reg\in\{LR\), \(DTR\), \(RFR\), \(SVMR\), \(MLPR\}\). To understand the processing of MN-SIFT to get Reg-SIFT, consider \(I_{F}\) and \(I_{G}\) be two images of the test set depicting the same scene in VS and IR bands, respectively. SIFT feature points are detected and MN-SIFT descriptors are computed. Let \(F_{w^{\prime}}\) and \(G_{z^{\prime}}\) be two sets of MN-SIFT descriptors that belong to \(I_{F}\) and \(I_{G}\) images, respectively, where \(w^{\prime}=1,2,3,...,w\) and \(z^{\prime}=1,2,3,...,z\). Total number of descriptors computed for \(I_{F}\) and \(I_{G}\) images are denoted as \(w\) and \(z\), respectively. Each \(F_{w^{\prime}}\) descriptor is processed through the regression model (i.e., testing) to obtain a Reg-SIFT descriptor (\(\overline{G}_{w^{\prime}}\)) and then the Reg-SIFT descriptors are matched with the MN-SIFT descriptors (\(G_{z^{\prime}}\)) of image \(I_{G}\). To understand the process, let \(F_{w^{\prime}}=[f_{w^{\prime}_{1}}\), \(f_{w^{\prime}_{2}}\), \(f_{w^{\prime}_{3}}\), \(\cdots\), \(f_{w^{\prime}_{m}}]\) be an MN-SIFT descriptor which is converted into a Reg-SIFT descriptor \(\overline{G}_{w^{\prime}}=[\overline{s}_{w^{\prime}_{1}}\), \(\overline{s}_{w^{\prime}_{2}}\), \(\overline{s}_{w^{\prime}_{3}}\), \(\cdots\), \(\overline{s}_{w^{\prime}_{m}}]\) as follows:  
\[T_{1} = [t_{1_{1}},\ t_{1_{2}},\ t_{1_{3}},\ \cdots,\ t_{1_{m}}], \tag{10}\]5 \[T_{1} = [t_{1_{1}},\ t_{1_{2}},\ t_{1_{3}},\ \cdots,\ t_{1_{m}}], \tag{10}\]6 (21) \[T_{1} = [t_{1_{1}},\ t_{1_{2}},\ t_{1_{3}},\ \cdots,\ t_{1_{m}}], \tag{10}\]7 (22) \[T_{1} = [t_{1_{1}},\ t_{1_{2}},\ t_{1_{3}},\ \cdots,\ t_{1_{m}}], \tag{10}\]8 \[T_{1} = [t_{1_{1}},\ t_{1_{2}},\ t_{1_{3}},\ \cdots,\ t_{1_{m}}], \tag{10}\]9  
After that, image matching using Reg-SIFT is carried out and the matching results are compared with state of the art descriptors. We use MN-SIFT for proposed Reg-SIFT because of its better robustness towards intensity and textual changes. MN-SIFT is based on MN features, which contain both local textural and structure information and performs well in cross spectral applications compared to NG-SIFT in [9]. NG-SIFT encapsulates only the structural information [15]. The experimental results of [9] show that MN-SIFT demonstrates better performance on multisensor images than SIFT, LC-SIFT, LBPG, DE-SIFT, and CS-LBP descriptors. Another reason for choosing MN-SIFT is its descriptor construction process, which is simple compared to that of EOH, LSS, MFD and HoDM descriptors. EOH uses Canny edges. The detection of Canny edges is relatively simple on VS images, but fails on LWIR/NIR images due to low contrast.