# A Method of Aerial Multi-Modal Image Registration
## 2 Real Scene Acquisition and Generation of Virtual Scene Simulation  
### Real Scene Imaging Analysis Based on Infrared Visible Binocular Vision  
For data acquisition of airborne multi-mode photoelectric sensor, the sensor position would be set near or parallel to the central axis of the aircraft as far as possible, such as the belly or just below the wing, so that the attitude data of the aircraft inertial navigation system, such as heading, pitch, roll, can also truly reflect the current state of the image sensor at the same time, and the two ones would be consistent as far as possible. At the same time, consider the installation space of the two types of photoelectric sensor equipment, and try to make the infrared thermal imaging and visible image imaging have the same field of vision. When installing the two types of photoelectric camera equipment, adopt the binocular vision mode of side-by-side installation, as shown in Figure 1.  
The coordinate system is established based on the optical axis of the camera and the relative displacement of the camera. The distance between cameras is \(L\), and the distance between the optical center of the camera and the farthest monitored target is \(D\). Then the effective field of view of each camera is expressed as \(V\):  
\[V=2 Dtan\ (\frac{\theta}{2}) \tag{1}\]  
where \(\theta\) is angle of view. For each camera, the ratio of its field of view overlapping range to its overall field of view is:  
\[r=\frac{V-L}{V}=1-\frac{L}{2 Dtan\ (\frac{\theta}{2})} \tag{2}\]  
Since the \(\theta\) is a fixed value when \(D=L\), the above equation is about 1, it can be considered that the two cameras have the same field of view. However, in practice, assuming  
Figure 1: Installation diagram of the binocular sensor. \(r>r_{min}\), we can make it possible to obtain the equivalent closest distance between two cameras. For example, if two cameras with a viewing angle of 60 degrees and a distance of 2 m are required to have at least 95% the same field of vision, then meters can be obtained, that is \(D>=34.64\), the observation target must be beyond this distance, otherwise, the images of the two cameras cannot be equivalent to similar fields of vision. If the installation positions of the two cameras are very close, from the perspective of scene imaging, it can be considered that the optical centers of the two cameras coincide, but the optical axes of the cameras do not overlap, so there is a certain angle difference. In this case, the difference between the two cameras can be described by affine transformation or an 8-parameter homography transformation matrix. The camera lens can perform translation, pan, tilt, rotation, zoom and other actions without restrictions. The 8-parameter homography projection transformation model can describe the camera's translation, horizontal scanning, vertical scanning, rotation, lens scaling and other movements, and the projection transformation can reflect the trapezoidal distortion during imaging, while the translation, rotation, rigid body and affine transformation can be regarded as special cases of projection transformation. Therefore, in most cases, the projection transformation model is used to describe the camera's movement or the coordinate transformation relationship between cameras in general motion. The projection transformation process of the camera model can be described by the following formula:  
\[Z_{c}\left[ \begin{array}{c}u\\ v\\ 1\end{array} \right]=\left[ \begin{array}{ccc}\frac{1}{dx}&0&u_{0}\\ 0&\frac{1}{dy}&v_{0}\\ 0&0&1\end{array} \right]\left[ \begin{array}{ccc}f&0&0&0\\ 0&f&0&0\\ 0&1&0\end{array} \right]\left[ \begin{array}{cc}R&t\\ 0^{T}&1\end{array} \right]\left[ \begin{array}{c}X_{w}\\ Y_{w}\\ Z_{w}\\ 1\end{array} \right] \tag{3}\]  
where in the homogeneous coordinate system, \([X_{w},Y_{w},Z_{w},1]\) is the world point coordinate, and \([u,v,1]\) is the pixel coordinate. \(dx\), \(dy\) represent the actual size of the pixel on the photosensitive chip, and \(u_{0}\), \(v_{0}\) are the center of the image plane.