# Heterogeneous Image Matching via a Novel Feature Describing Model
## 3 Proposed Method
### Main Steps  
Based on previous sections, a novel frame for heterogeneous image matching can be formulated as follows. 1. Gradient histogram and adaptive intervals. According to the matching task, some prior information of keypoints can be helpful to determine proper candidate intervals for computing gradient histogram. Such adaptive intervals are more advantageous to describe the keypoints accurately and distinguish different keypoints than traditional uniform eight-intervals. The local gradient information of some known keypoints can be integrated as \(\
abla I(x),x\in\Omega_{key}\). \(\Omega_{key}\) means the set of keypoints and \(\
abla I(x)\) is the gradient at a keypoint \(x\) in the rotated local patch. Then, refined interval statistics is carried out on it, and a weighted histogram can be denoted as  
\[h_{i}=\sum_{|\theta(x)-c_{i}|<\frac{m}{m}}|\
abla I(x)|,\ i=1,2,\cdots,m. \tag{13}\]  
Here, \(\{c_{i}\}_{i=1}^{m}\) means the centers set of candidate intervals. \(\theta(x)\) denotes the direction angle of \(\
abla I(x)\). Figure 4 shows the traditional histogram and the weighted version of Lena image as \(m=36\). Then, an optimization model (14) is introduced to determine a set of adaptive intervals for local gradient statistics:  
Figure 3: Two types of heterogeneous images: (**a**) original; (**b**) heterogeneous type 1; (**c**) heterogeneous type 2. \[ \begin{array}{ll}\min_{k,C_{j}}&\frac{1}{k}\sum_{j=1}^{k}\ \big{(}\sum_{c_{i}\in E_{j}}h_{i}-\frac{\sum_{i}h_{j}}{k} \big{)}^{2},\ \max_{k,C_{j}}\ \ \ \ \ \frac{1}{k}\sum_{j=1}^{k}\frac{\sum_{c_{j}\in E_{j}^{h}}h_{i}}{\sum_{c_{i}\in E_ {j}}h_{i}},\\ \text{s.t.}& E_{1}=\begin{cases}[C_{1}-\frac{\epsilon_{0}}{2},C_{1}+\frac{ \epsilon_{1}}{2}],&C_{1}+C_{k}>\pi,\\ [C_{k}+\frac{\epsilon_{0}}{2},\frac{3\pi}{2}]\cup[-\frac{\pi}{2},C_{1}],&C_{1}+ C_{k}\leq\pi,\\ \end{cases}\\ & E_{k}=\begin{cases}[C_{k},\frac{3\pi}{2}]\cup[-\frac{\pi}{2},C_{1}-\frac{ \epsilon_{0}}{2}],&C_{1}+C_{k}>\pi,\\ [C_{k}-\frac{\epsilon_{k-1}}{2},C_{k}+\frac{\epsilon_{0}}{2}],&C_{1}+C_{k}\leq \pi,\\ \end{cases}\\ & E_{j}=[C_{j}-\frac{\epsilon_{j-1}}{2},C_{j}+\frac{\epsilon_{j}}{2}],E_{j}^{h }=[C_{j}-\frac{\epsilon_{j-1}}{4},C_{j}+\frac{\epsilon_{j}}{4}],\ j=2,3,\cdots,k-1,\\ &\epsilon_{0}=\frac{2\pi+C_{1}-C_{k}}{2},\ e_{j}=C_{j+1}-C_{j},\ j=1,2,\cdots,k-1,\\ &\frac{-\pi}{2}\leq C_{j}<C_{j+1}\leq\frac{3\pi}{2},\ j=1,2,\cdots,k-1.\end{array}  \tag{14}\]  
Here, \(E_{j}(j=1,2,\cdots,k)\) means the adaptive intervals and \(C_{j}(j=1,2,\cdots,k)\) denotes the centers, \(k\) is the number of selected intervals. It can be approximately solved by a dynamic programming algorithm after \(k\) preset. 2. SIFT keypoints localization. In this step, the keypoints are detected as the SIFT. The extreme points in the DoG space can be detected by non-maximum suppression. Let \(D(x)\) and \(U_{r}(x)\) denote the DoG function and r-neighborhood of \(x\); then, the keypoints can be located as Equations (5) and (6). 3. Main direction computing based on PCA of local patch. The main direction will be computed on the local gradient field of patch \(A(x)\) centered the keypoint \(x\). To recognize heterogeneous image features, we normalize the keypoint a maximal point in the difference space. The local gradient field extracted from the pre-computed global gradient field will be treated in the same way. Then, PCA will be carried out on the normalized rearranged \(2\times n\) matrix \(L\), and the first component can used to determine the main direction:  
\[\theta(A)=\arctan\frac{v_{1}[2]}{v_{1}[1]}. \tag{15}\]  
Here, \(v_{1}=\text{PCA}(1,\
abla A)\) means the first component of PCA carried out on \(\
abla A\); it can be denoted in more detail as  
\[\lambda_{1}=\max_{\lambda_{i}}\Big{\{}|\lambda_{i}|,LL^{T}v_{i}=\lambda_{i}v_{ i},i=1,2\Big{\}}\,\ L=\frac{\
abla I_{loc}-mean(\
abla I_{loc})}{std(\
abla I_{loc})}.\]  
Figure 4: Gradient field statistics: (**a**) a gradient field on Lena image; (**b**) a traditional histogram; (**c**) the weighted histogram.