# A Method of Aerial Multi-Modal Image Registration
## 3 Multi-Modal Image Registration Based on Virtual Scene ROI Drive
### Curvature Based Registration Method
#### 3.2.1 Automatic Threshold Canny Edge Detection Based on Virtual and Real Fusion  
Literature [34] proposes a curvature-based registration method for infrared and visible images, which effectively solves the problem of heterogeneous image matching with the help of geometric deformation. The registration algorithm first detects the edge of the image according to the Canny algorithm, then tracks the contour, calculates the curvature of the contour points, and considers that the position of the local minimum curvature is the feature point. Subsequently, the high-dimensional feature description vector is obtained by imitating the feature descriptor of SIFT, so as to realize the registration of multi-modal images.  
In Canny edge detection, the determination of automatic threshold is based on an important parameter, that is the proportion of pixels in an image that are not edges to the image, denoted by \(\eta\). This parameter needs to be given by the prior experience of the image. Generally, it is estimated that \(\eta=0.7\). In this paper, according to the virtual image of the scene, the \(\eta\) can be roughly estimated, so that Canny edge detection with an automatic threshold can be realized. The specific steps are as follows:  
1. Smooth image using Gaussian filtering: \[I(x,y)=I(x,y)*F_{\sigma}(x,y),1\leq x\leq W,1\leq y\leq H\] (14) where \(I(x,y)\) is the grayscale intensity image converted from the truecolor image, scaled the display based on the range of pixel values, \"*\" denotes the convolution operation and \(F_{\sigma}(x,y)\) is the surround function: \[F_{\sigma}(x,y)=\kappa e^{-\frac{(x+y)^{2}}{e^{2}}}\] (15) where \(\sigma\) is the Gaussian surround space constant, and \(\kappa\) is selected such that: \[\int\int F_{\sigma}(x,y)dxdy=1\] (16)
2. Calculate gradients using a derivative of the Gaussian filter: Compute smoothed numerical gradient of the image \(I\) along \(x\), \(y\) direction, denoted by \(I_{x}\), \(I_{y}\), respectively.
3. Compute the magnitude of gradient: \[M(x,y)=\sqrt{I_{x}^{2}(x,y)+I_{y}^{2}(x,y)}\] (17)
4. Create a histogram with \(N\) bins for the magnitude \(M\): Initialize: \(num[j]=0\), \(j=1\),..., \(N\).  
Figure 4: ROI region generation based on flight parameters.  
Traverse: \(num[j]=num[j]+1\), where \(M(x,y)\in[m+(j-1)d,m+jd]\), \(m=\min\limits_{x,\,y}\{M(x,y)\}\), \(M=\max\limits_{x,\,y}\{M(x,y)\}\), \(d=\frac{M-m}{N}\), for each \(x,y\), \(i\).
5. Find the smallest h that meets the condition: \(\sum_{i=1}^{h}num(i)>=\eta\cdot m\cdot n\).
6. Select \(h\) as the optimal high threshold, \(h\cdot r\) as the optimal small threshold, where \(r\) is pre-defined proportional value.  
2.2 Infrared and Visible Image Registration Based on Contour Angle Orientation and Virtual Reality Fusion  
Inspired by CAO, this paper proposes an improved CAO algorithm based on virtual and real fusion. According to the region of interest determined by Section 3.1, first, calculate the automatic threshold of the Canny edge detection operator, then track the contour, and compute the curvature of the contour points, and finally determine the feature points. Due to the visible image under low visibility considered in this paper, the edge blur affects the accuracy of matching to some extent, which is easy to lead to wrong matching point pairs. The virtual image proposed in this paper can be used as effective reference information for matching. Equation (13) gives the approximate corresponding position of the infrared image feature points in the visible image, thus realizing the process from rough matching to fine matching.