# A Method of Aerial Multi-Modal Image Registration
## 3 Multi-Modal Image Registration Based on Virtual Scene ROI Drive
### ROI Region Generation Based on Virtual Scene Driving  
Since we know the location of the airport surface area in the virtual scene, we consider that the key object in the approach phase is the airport area, especially the runway, taxiway and other areas, and the image registration and fusion effect of the above airport area is the key object in the air approach. Therefore, in order to reduce the impact of other imaging areas on the airport area registration, this paper is based on the flight parameter drive OSG virtual scene simulation. Before the simulation, adjust the resolution, imaging depth and other relevant parameters of the virtual scene to be roughly consistent with the imaging distribution of the photoelectric equipment scene. Assume the coordinates \([x,y,z]^{\intercal}\) of a point \(P\) in a 3D scene. The camera position error \([\Delta x,\Delta y,\Delta z]^{\intercal}\) caused by the relative relationship between the object point and the camera and the GPS error is equivalent to the offset of the space point in the 3D model. Combined with camera attitude error \([\Delta\alpha,\Delta\beta,\Delta\gamma]^{\intercal}\) caused by AHRS, the position of the point \(P\) in the camera coordinate system is  
\[p^{\prime}=\left[ \begin{array}{ccc}cos\ \Delta\gamma&-sin\ \Delta\gamma&0\\ sin\ \Delta\gamma&cos\ \Delta\gamma&0\\ 0&0&1\end{array} \right]\left[ \begin{array}{ccc}cos\ \Delta\beta&0&sin\ \Delta\beta\\ 0&1&0\\ -sin\ \Delta\beta&0&cos\ \Delta\beta\end{array} \right]\left[ \begin{array}{ccc}1&0&0\\ 0&cos\ \Delta\alpha&-sin\ \Delta\alpha\\ 0&sin\ \Delta\alpha&cos\ \Delta\alpha\end{array} \right]\left[ \begin{array}{c}x+ \Delta x\\ y+\Delta y\\ z+\Delta z\end{array} \right] \tag{10}\]  
Expand (10) to get  
\[\left\{ \begin{array}{l}s_{x}=\frac{f[(x+\Delta x)cos\ \Delta\beta-cos\ \Delta\gamma+(y+\Delta y)(cos\ \Delta\alpha-sin\ \Delta\gamma+sin\ \Delta x-sin\ \Delta\beta-cos\ \Delta\gamma)]\pm f(z+\Delta z)(sin\ \Delta\alpha-sin\ \Delta\gamma+cos\ \Delta\alpha-cos\ \Delta\beta-cos\ \Delta\gamma)}{-(x+\Delta x)sin\ \Delta\beta+(z+\Delta z)cos\ \Delta\alpha-cos\ \Delta\beta+(y+\Delta y)sin\ \Delta\alpha-cos\ \Delta\beta+(y+\Delta y)sin\ \Delta\alpha-cos\ \Delta\beta}{\Delta\beta+(y+\Delta y)sin\ \Delta\alpha-cos\ \Delta\beta}\\ s_{y}=\frac{f[(x+\Delta x)cos\ \Delta\beta-sin\ \Delta\gamma+(y+\Delta y)(cos\ \Delta\alpha-cos\ \Delta\gamma+sin\ \Delta\alpha-sin\ \Delta\beta-sin\ \Delta\gamma)]-f(z+\Delta z)(sin\ \Delta\alpha-cos\ \Delta\gamma-cos\ \Delta\alpha-sin\ \Delta\beta-sin\ \Delta\gamma)}{-(x+\Delta x)sin\ \Delta\beta+(z+\Delta z)cos\ \Delta\alpha-cos\ \Delta\beta+(y+\Delta y)sin\ \Delta\alpha-cos\ \Delta\beta}\\ \end{array} \right. \tag{11}\]  
where \(f\) is the focal length. According to the equipment error, the attitude angle error is \(0.5^{\circ}\), so in (11), \(cos\ (\theta)\ \approx\ 1\), \(sin\ (\theta)\ \approx\ 0\ (\theta=\Delta\alpha,\Delta\beta,\Delta\gamma)\), substituting (11):  
\[\left\{ \begin{array}{l}s_{x}^{\prime}=\frac{f(x+\Delta x)}{z+\Delta z}\\ s_{y}^{\prime}=\frac{f(y+\Delta y)}{z+\Delta z}\end{array} \right. \tag{12}\]  
Henceforth the approximate expression of the vertex deviation of virtual and real runways is obtained from (11), and (12):  
\[ \begin{array}{l}(\Delta s)_{x}=\frac{f(z\Delta x-\Delta x)}{z(z+\Delta z)}\\ (\Delta s)_{y}=\frac{f(z\Delta y-y\Delta z)}{z(z+\Delta z)}\end{array}  \tag{13}\]  
where \(\Delta x\), \(\Delta y\) and \(\Delta z\) is related to position accuracy. In different approach and landing phases, different areas are selected as ROI. During the phase far away from the runway, the area outside the runway, such as roads, fields, buildings and other dense areas, is selected as ROI, as effective feature points cannot be extracted from the blurred runway in the visible image. For the phase approaching the runway, the runway and adjacent areas are selected as ROI. According to the above selection criterion, more features can be extracted in the region of interest for matching, as shown in Figure 4, and the area below the dotted line is selected as ROI.