# Hybrid Multimodal Medical Image Fusion Method Based on LatLRR and ED-D\({}^{2}\)GAN
## 2 Proposed Method
### Salient Region Images Fusion Method  
Salient region images can reflect important information, such as the edge contour and texture details of the source images. Therefore, the selection of the salient region fusion rule is directly related to the sharpness and edge intensity of the fused image. Considering that the local features of an image are not expressed by a single pixel, but by multiple pixels in a local region, and how each pixel has a strong correlation with each other, the simple weighted fusion rule based on a single pixel cannot reflect the feature information of the region well. However, the fusion rule based on regional energy can overcome the one-sidedness of the above methods and obtain more local feature information of the images. Therefore, the fusion method based on regional energy [23] was selected as the basis of the salient region fusion rule in this paper. We considered that when the energy of two local regions is relatively similar, the relative loss of information is easy to be caused by directly selecting the pixel value with more regional energy. Therefore, a threshold adaptive weighted fusion algorithm based on regional energy ratio is proposed in this paper. According to the constant changes of regional center pixel and its corresponding regional energy, the weight matrix was set to adjust the weight coefficient adaptively, so as to achieve the purpose of fully preserving the details of the fused image. The fusion processes are as follows. Firstly, the salient region images \(CT\_S1\) and \(PET\_S2\) decomposed by LatLRR were segmented by the sliding window, respectively, and the regional energy \(E_{CT}(m,n)\) and \(E_{PET}(m,n)\) of the pixel centered on \((m,n)\) were obtained. The calculation formulas are as follows:  
\[E_{CT}(m,n)=\sum_{i\in S}\sum_{j\in T}W(m,n)\times[CT\_S1(i+m,j+n)]^{2}, \tag{8}\]  
\[E_{PET}(m,n)=\sum_{i\in S}\sum_{j\in T}W(m,n)\times[PET\_S2(i+m,j+n)]^{2}, \tag{9}\]  
where \((i,j)\) is the relative offset of pixels in the region window from the center pixel. \(S\) and \(T\) are the maximum row and maximum column coordinates of the regional window. \(W\) is a weight matrix of size \(3\times 3\). The normalized \(W\) is defined as:  
\[W=\frac{1}{16} \begin{bmatrix}1&2&1\\ 2&4&2\\ 1&2&1\end{bmatrix} , \tag{10}\]  
Then, the regional energy ratio \(E_{ratio}(m,n)\) is obtained according to the regional energy, and the calculation formula is as follows:  
\[E_{ratio}(m,n)=\frac{E_{PET}(m,n)}{E_{CT}(m,n)}, \tag{11}\]  
Finally, the salient region fused image \(I_{sf}\) is calculated by weighting method, and the corresponding formula is as follows:  
\[I_{sf}(m,n)=w_{1}\times PET\_S2(m,n)+w_{2}\times CT\_S1(m,n), \tag{12}\]where \(w_{1}\) and \(w_{2}\) are the weighting coefficients. The specific calculation formula is as follows:  
\[ \begin{bmatrix}w_{1}\\ w_{2}\end{bmatrix} = \begin{cases}\begin{bmatrix}0&1\end{bmatrix}^{T}&E_{ratio}<th1 \\ \begin{bmatrix}E_{PET}/(E_{PET}+E_{CT})\\ E_{CT}/(E_{PET}+E_{CT})\end{bmatrix}&th1<E_{ratio}<th2,\\ \begin{bmatrix}1&0\end{bmatrix}^{T}&E_{ratio}>th2\end{cases}  \tag{13}\]  
where \([\cdot]^{T}\) is the matrix transpose operator, and \(th1\) and \(th2\) are threshold coefficients determined according to the overall energy distribution of the images. When the energy ratio of the region is too small or too large, the weight of the region with higher energy is set to 1 and the weight of the region with lower energy is set to 0. When the energy ratio of the region is within the threshold range, the adaptive weight is calculated based on the energy ratio. That is, the larger the region energy, the larger the corresponding weighting coefficient, and the higher the proportion in the fused image. On the contrary, the smaller the region energy, the lower the proportion in the fused image.