# 2D/3D Multimode Medical Image Registration Based on Normalized Cross-Correlation
## 1 Introduction  
Image-guided surgery, which involves computer vision, biomedicine, imaging, automatic control, and other disciplines, is an interdisciplinary research direction [1, 2, 3, 4, 5]. Through the comprehensive application of a variety of medical image information, it carries out the preoperative diagnosis, disease analysis, planning of surgical path, intraoperative localization of the lesion, real-time tracking of surgical instruments, and adjustment of the spatial position of surgical instruments to achieve an accurate diagnosis [6, 7]. Image navigation surgery's success largely depends on the registration accuracy of preoperative image data and intraoperative image data and the accuracy of the 2D/3D registration algorithm. Image registration is developed for the integration of multisource image information. Image registration's primary purpose is to find the spatial transformation mapping relationship between two or more images in the same location to obtain the maximum image information [8]. Although there are many research and clinical applications, the image registration procedures in image navigation surgery still need further improvement [9, 10, 11, 12, 13]. Therefore, developing more advanced registration methods is necessary to accurately and effectively register medical images. Two-dimensional/three-dimensional registration can be divided into pseudo-multimodal registration and multimodal registration. Generally, 3D CT imaging is used before the operation, and 2D X-ray imaging is used during operation [7, 14]. Although the imaging principles of the two imaging methods are almost the same, the differences of the photon energy and detector characteristics and radiation and artifact will cause the difference of image gray [15, 16]. MRI is the preoperative imaging method, while X-ray image is the intraoperative imaging mode [8]. Three-dimensional images and two-dimensional imagesmust be unified into the same dimension to perform 2D/3D registration. By converting 3D images into 2D images or 2D images into 3D images, the former generates 2D/2D registration, while the latter generates 3D/3D registration [17; 18]. More specifically, we can achieve dimensional unification through projection, back-projection, or reconstruction strategies [19; 20]. The advantage of projection-based methods is that they only need little or no segmentation [21; 22], so they hardly depend on the segmentation algorithm's accuracy [23; 24]. In addition, only one 2D image may be enough to achieve 2D/3D registration in the projection and back-projection strategy. However, the reconstruction method requires at least two or more 2D images to reconstruct 3D images with high enough accuracy to achieve accurate and robust 3D/3D registration for high-resolution 3D images collected before surgery [4; 25; 26]. In general, the greater the number of intraoperative 2D images, the higher the registration accuracy. However, in clinical practice, due to a long time of image acquisition and reconstruction and the limitation of radiation dose on human health, it is unrealistic to obtain multiple X-ray images simultaneously during the operation [22; 27; 28]. However, because a GPU can offset the projection algorithm's time cost and other hardware-accelerated calculations, it is possible to realize clinical 2D/3D image registration by using the projection strategy [29]. There are a few research pieces on 2D/3D medical image registration using deep learning in recent years. Generally speaking, there are two strategies in the current literature [30]. The first is to estimate the similarity between images using a deep learning network instead of the traditional similarity measure, thus driving the iterative optimization strategy to optimize gradually. The second is to directly obtain the spatial transformation parameters through the end-to-end deep regression network without iterative execution. For example, [PERSON] [30; 31; 32; 33; 34] directly predicted the spatial mapping relationship between two images using an end-to-end depth network.