# Hybrid Multimodal Medical Image Fusion Method Based on LatLRR and ED-D\({}^{2}\)GAN
## 2 Proposed Method
### Low-Rank Region Images Fusion Method
#### 2.1.3 Loss Function  
The model architecture of ED-D\({}^{2}\)GAN consists of three parts: generator \(G\), discriminator \(D_{CT}\) and discriminator \(D_{PET}\). Therefore, the loss function also includes three parts, that is, generator loss function \(L_{G}\), discriminator \(D_{CT}\) loss function \(L_{D_{CT}}\) and discriminator \(D_{PET}\) loss function \(L_{D_{PET}}\). 1. Loss function of generator  
Since the training process of the basic GAN model is unstable, this paper uses content loss to impose additional constraints on the generator. The loss function of a generator consists of the content loss between the generated image and the low-rank region images of the source images and the adversarial loss between the generator and the discriminator, which is defined as follows:  
\[L_{G}=L_{G}^{adv}+\alpha L_{c}, \tag{1}\]  
where \(L_{G}^{adv}\) is the adversarial loss, \(L_{c}\) is the content loss, and \(\alpha\) is the parameter that controls the trade-off. \(L_{G}^{adv}\) directs the generator to generate a real fused image through an adversarial game between the generator and discriminator in order to fool both two discriminators, defined as follows:  
\[ \begin{array}{ll}L_{G}^{adv}&=E(\log(1-D_{CT}(G(CT\_L1\_E,PET\_L2))))+\\ &E(\log(1-D_{PET}(G(CT\_L1\_E,PET\_L2))))),\end{array}  \tag{2}\]  
where \(D_{CT}\) and \(D_{PET}\) respectively represent two discriminators, \(G(CT\_L1\_E,PET\_L2)\) represents the fused image, and \(E\) represents the mathematical expected value, that is, the generator expects the generated fused image to deceive the discriminators. \(L_{c}\) constrains the content similarity between the generated image and the low-rank region images of the source images, so that the fused image can retain more effective information of the source images. Since the texture details of CT images are mainly characterized by gradient changes, the functional information of PET images can be characterized by pixel intensity. Therefore, \(L_{c}\) mainly includes gradient loss and intensity loss. The specific definition is as follows:  
\[L_{c}=L_{grad}+\beta L_{in}, \tag{3}\]  
where \(L_{grad}\) represents gradient loss, \(L_{in}\) represents intensity loss, and \(\beta\) is the parameter controlling the trade-off. \(L_{grad}\) is used to measure the retention degree of texture details in the fused image. In order to retain finer texture information in the fused image, this paper constructs gradient loss according to the principle of maximum selection, which can not only enhance the preservation of texture detail information, but also effectively prevent the diffusion of edge detail information in high-contrast areas. \(L_{grad}\) is defined as follows:  
\[L_{grad}=\Big{|}\Big{|}\text{max}\Big{(}\Big{|}\
abla^{2}CT\_L1\_E\Big{|}, \Big{|}\
abla^{2}PET\_L2\Big{|}\Big{)}-\Big{|}\
abla^{2}G(CT\_L1\_E,PET\_L2) \Big{|}\Big{|}\Big{|}\Big{|}\Big{|}, \tag{4}\]  
where \(|\cdot|\) represents the absolute value function, \(||\cdot||_{1}\) is the L1 norm, \(\text{max}(\cdot)\) is a maximum function, \(\
abla^{2}\) is the Laplacian gradient operator. \(L_{in}\) is used to constrain the fused image to maintain a similar intensity information distribution with the source images, thus maintaining significant contrast information. \(L_{in}\) is defined as follows:  
\[ \begin{array}{ll}L_{in}=\gamma||G(CT\_L1\_E,PET\_L2)-PET\_L2||_{F}^{2}+\\ (1-\gamma)||G(CT\_L1\_E,PET\_L2)-CT\_L1\_E||_{F}^{2},\end{array}  \tag{5}\]  
where \(||\cdot||_{F}^{2}\) is the _Fronbenius_ norm, and \(\gamma\) is the parameter that controls the trade-off. 2. Loss function of discriminator  
In this paper, two independent discriminators \(D_{CT}\) and \(D_{PET}\) are used to constrain the generator to capture more contrast information and texture information, respectively.