# Hybrid Multimodal Medical Image Fusion Method Based on LatLRR and ED-D\({}^{2}\)GAN
## 1 Introduction
introduces the Generative Adversarial Network (GAN) [16] into the image fusion field, establishing an adversarial game between the generator and discriminator, which can obtain fused images with outstanding target information. [PERSON] et al. [17] realized end-to-end anatomical and functional medical image fusion based on GAN and obtained fused images with clear edges and rich details. However, these methods only use one discriminator, which makes it easy to cause losses of effective information in the fused images, and fusion methods based on deep learning often pay little attention to noise-processing of the source images. In addition, it is still a challenge to achieve effective fusion by designing network architectures and loss functions compared to the methods based on multi-scale decomposition. Considering the advantages and limitations of a single fusion method, researchers have proposed some methods based on hybrid models to apply deep learning to the framework of traditional image fusion tasks to further improve the quality of fused images. Latent low-rank representation (LatLRR) [18] is usually used in clustering analysis tasks, which can remove the noise region contained in the source images and extract the global and local structure of data. [PERSON] et al. [19] combined LatLRR with CNN, used LatLRR and Rolling Guided Image Filtering (RGIF) to decompose the source images at two levels, and used the fusion rules based on CNN to fuse the detailed layers, thus improving the contrast and sharpness of the fused images. [PERSON] et al. [20] proposed a multi-modal medical image fusion method based on multi-scale transformation and the deep stacked convolutional neural network (DSCNN). In this method, the trained DSCNN model was used to decompose the source images into low-frequency and high-frequency images, and were then fused respectively. The proposed method can adaptively decompose and reconstruct the image in the fusion process. However, the image fusion methods based on CNN have a strong dependence on the quality of the source images. Due to the particularity of medical images, there is a small number of registered medical images in the current public datasets. Therefore, the application of the image fusion method based on CNN in the field of multimodal medical image fusion has certain limitations. [PERSON] et al. [21] proposed a medical image fusion method based on GAN and shift-invariant shearlet transform (SIST). In this method, SIST was used to decompose the source images, and the trained GAN model was used as the fusion rule of the high-pass sub-band, and the low-pass sub-band fusion was achieved by local energy-weighted summation and bilateral filter. It can effectively suppress the phenomena of artifacts and distortions in the fused images, but it is easy to lose some details in the source images because only one discriminator is being used. Based on the above problems, in order to better preserve the anatomical structure and contour intensity information of the CT source images, the functional and metabolic information of the lesion area in the PET source images are highlighted, and to enhance the visibility of the source images, a hybrid multimodal medical image fusion method (LatLRR-GAN) based on LatLRR and dual discriminators GAN (ED-D\({}^{2}\)GAN) is proposed by combining LatLRR and GAN in this paper. Considering the denoising ability of LatLRR, in order to reduce the impact of noise in the fusion process, this paper firstly uses LatLRR to decompose the source images. Secondly, we show how GAN has the following advantages: (1) Strong feature extraction ability; (2) an end-to-end image fusion process; and (3) how the quality of fused images can constantly adjust through the adversarial game between generator and discriminator. This paper introduces GAN into the framework of multimodal medical image fusion tasks based on decomposition transformation to alleviate the shortage of manual design of complex fusion rules and improve the quality of the fused images.