# Feature Point Matching Method Based on Consistent Edge Structures for Infrared and Visible Images
## 2 Proposed Methods
### Common Feature Point Detection  
Non-common feature points reduce the detection repeatability of feature points. Because local details of infrared and visible images are different from mutual missing and disparate position responses, the feature points extracted on local details are non-common feature points. The performance of feature point matching methods depend on the repeatability of detected feature points [10; 17]. Matching of non-common feature points of images can only bring in the wrong matches, which will bring down the matching performance.  
The proposed common feature point detection method utilizes the consistent edges of infrared and visible images to detect feature points. Due to the similarity of consistent edge structures, the percentage of common feature points in the detected feature points can be increased. The common feature point detection method contains two steps. First, candidate common feature points and consistent edges of infrared and visible images are detected. Second, consistent edges are leveraged to check whether the candidate feature points are common feature points or not.  
Candidate common feature points are detected by the Harris response [41] of images. [PERSON] et al. [17] prove by experiments that Harris corners have better repeatability than other feature point detection methods for infrared and visible images. Harris Response \(R\) of the point \(p\) is computed by the equations below.  
\[M(p)=\sum_{(x,y)\in N(p)}w(x,y)\left[\begin{array}{cc}I_{x}^{2}&I_{x}I_{y}\\ I_{x}I_{y}&I_{y}^{2}\end{array}\right] \tag{1}\]  
\[R(p)=det(M(p))-k\cdot\left(trace(M(p))\right)^{2} \tag{2}\]  
where \(w\) is the Gaussian weight function, \(I_{x}\) and \(I_{y}\) are image derivatives, \(N(p)\) is the \(3\times 3\) neighborhood of point \(p\), and \(k\) is a constant in the interval [0.04, 0.06] [39]. Points with response \(R>0\) are marked as candidate feature points [39]. The set of candidate common feature points \(P^{\prime}\) is obtained by local maximum suppression of candidate feature points.  
Common feature points of infrared and visible images are obtained by filtering the feature point set \(P^{\prime}\), according to the constraints of the consistent edges set \(le\). The set of common feature points denoted as \(P\) is defined by the equation below.  
\[P=\left\{p_{i}\middle|p_{i}\in P^{\prime}\wedge\ \exists p_{j}\in N(p_{i}) \wedge\ \exists e_{k}\in le\wedge p_{j}\in e_{k}\right\} \tag{3}\]  
where \(N(p)\) is also the \(3\times 3\) neighborhood. According to Equation (3), the feature points located on or near the consistent edge are selected as the common feature points. Because the consistent edge reflects the global structure, which is consistent in infrared and visible images, feature points constrained by the consistent edge can be considered common feature points.  
An example of common feature point detection is shown in Figure 2. As shown in Figure 2a, the feature points generated by the text on the visible image have no corresponding points on the infrared image. As a result, these feature points are non-common feature points. Matching of these feature points can only get the wrong matches. The feature points extracted by the common feature point detection method are shown in Figure 2b. The number of feature points extracted on the text of visible images is reduced significantly when compared to Figure 2a. The common feature point detection method can improve the matching performance by removing non-common feature points.