# Feature Point Matching Method Based on Consistent Edge Structures for Infrared and Visible Images
## 3 Experimental Results
### Datasets and Evaluation Criteria  
Two public datasets, known as the CVC (Computer Vision Center of Universitat Autonoma de Barcelona) dataset [27] and LWIR (Long Wave Infrared Images) dataset [31], are utilized to validate the performance of DDCE. These two datasets are composed of visible and long wave infrared images. Homography transformations between images are provided by the datasets. Parameter \(\alpha\) is utilized in feature detection and description of DDCE. For the CVC dataset and the LWIR dataset, \(\alpha\) is set as 20.  
Feature detection repeatability, feature matching accuracy, and RANSAC (RANdom SAmple Consensus) [44] estimation result are used to evaluate the matching performance of DDCE. Feature detection repeatability depicts the percentage of repeatable features detected in infrared and visible  
\begin{table}
\begin{tabular}{l} \hline \hline
**Input:** infrared image \(I_{ir}\), visible image \(I_{vis}\) \\
**Output:** feature correspondence \(C\) \\ \hline
1. // (1)Consistent Edge Extraction \\
2. \(E[\epsilon_{1}\epsilon_{2},\ldots,\epsilon_{M}]=\) Canny(\(I\)) \\
3. \(le=\left\{e_{i}\right\}\left|e_{i}\right|>\alpha,e_{i}\epsilon E\) \\
4. // (2) Common Feature Detection \\
5. \(P^{\prime}=\) Harris\(\_\)Corner(\(I\)) \\
6. \(P=\left\{p_{i}\right|p_{i}\in P^{\prime}\wedge\exists p_{j}\in N(p_{i})\wedge \exists e_{k}\in le\wedge p_{j}\in e_{k}\right\}\) \\
7. // (3) Feature Description Establishment \\
8. \(f=\) Edge\(\_\)Orientation\(\_\)Histogram(\(P\), \(E\)) \\
9. // (4) Feature Matching \\
10. \(C=\) Match(\(f_{ir},f_{vis}\)) \\ \hline \hline \end{tabular}
\end{table}
Table 1: Algorithm procedure.  
images. The re-projection error \(\varepsilon\) of two features with positions \(p_{i}\) and \(p_{j}\) can be expressed by the equation below.  
\[\|p_{i}-Hp_{j}\|=\varepsilon, \tag{9}\]  
where \(H\) is the homography transformation. The repeatability of two features is computed by the formula below.  
\[C(i,j)=\left\{\begin{array}{ll}1&\varepsilon<2\\ 0&\text{otherwise}\end{array}\right., \tag{10}\]  
where the threshold of the re-projection error \(\varepsilon\) is 2 pixels [17]. Repeatability is derived from the ratio between the number of repeatable features and the minimum number of features in two images.  
\[Repeatability=\frac{N_{a}}{min(\#I_{vis},\#I_{ir})}, \tag{11}\]  
where \(N_{a}\) is the number of all repeatable features.  
\[N_{a}=\sum_{i}\sum_{j}C(i,j), \tag{12}\]  
and #I is the number of features detected in image \(I\).  
Precision, recall, and the F1 score are used to assess feature matching accuracy. The correspondence of two features can be formulated by the equation below.  
\[M(i,j)=\left\{\begin{array}{ll}1&f_{i},\ f_{j}\ \text{is matched}\\ 0&\text{otherwise}\end{array}\right., \tag{13}\]  
A pair of points \(p_{i}\) and \(p_{j}\) can be identified as a correct match if they satisfy Equation (7) and Equation (10) simultaneously. Precision is computed by the formula below.  
\[Precision=\frac{N_{c}}{N_{c}+N_{w}}, \tag{14}\]  
where \(N_{c}\) is the number of correct matches  
\[N_{c}=\sum_{i}\sum_{j}C(i,j)\times M(i,j), \tag{15}\]  
and \(N_{w}\) is the number of wrong matches.  
\[N_{w}=-N_{c}+\sum_{i}\sum_{j}M(i,j), \tag{16}\]  
Recall is computed by the formula below.  
\[Recall=\frac{N_{c}}{N_{a}}, \tag{17}\]  
The F1 score is computed by the equation below.  
\[\text{F1}=2\times(\text{Precesion}\times\text{Recall})/(\text{Precesion}+\text{ Recall}). \tag{18}\]  
RANSAC is used to estimates the homography transformation and identifies the correct matches because of the existence of wrong matches. As shown in Reference [28], the matching image formed by identified correct matches is used to display the result of RANSAC estimation.