# Morphological Component Analysis-Based Perceptual Medical Image Fusion Using Convolutional Sparsity-Motivated PCNN

Authors: Chuangeng Tian; Lu Tang; Xiao Li; Kaili Liu; Jian Wang
Journal: Scientific Programming
 (2021)

---

## Abstract

This paper proposes a perceptual medical image fusion framework based on morphological component analysis combining convolutional sparsity and pulse-coupled neural network, which is called MCA-CS-PCNN for short. Source images are first decomposed into cartoon components and texture components by morphological component analysis, and a convolutional sparse representation of cartoon layers and texture layers is produced by prelearned dictionaries. Then, convolutional sparsity is used as a stimulus to motivate the PCNN for dealing with cartoon layers and texture layers. Finally, the medical fused image is computed via combining fused cartoon layers and texture layers. Experimental results verify that the MCA-CS-PCNN model is superior to the state-of-the-art fusion strategy.


---

## Body

## 1. Introduction

In clinical applications, medical images include anatomical images and functional images. Anatomical images provide information of dense structures [1], for instance, X-ray computed tomography (CT) and magnetic resonance imaging (MRI). Functional images reflect information of blood flow and blood activity [2], for instance, positron emission CT (PET) and single-photon emission CT (SPECT). Medical images with single modality do not provide sufficient information in diagnosing diseases; medical image fusion (MIF) technology provides an effective method via merging medical images with different modalities into a comprehensive MIF image to aid radiologists for better diagnosis [3–5].Many MIF algorithms have been addressed in the last dozen years. These methods include the multiscale decomposition- (MSD-) based fusion strategy [6–10], sparse representation- (SR-) based fusion strategy [11], and pulse-coupled neural network- (PCNN-) based fusion strategy [12, 13]. To pursue satisfactory fusion performance, attempts were made to use the PCNN based on MST [14–16]. PCNN is a cat visual cortex biologically inspired neural network, which is used in medical image fusion. Huang et al. [17] integrated non-subsampled contourlet transform (NSCT) with the PCNN for SPECT and CT image fusion. Non-subsampled shearlet transform (NSST) was combined with the PCNN to fuse medical images [18]. However, NSCT- or NSST-based fusion strategy has high computational complexity due to proper contours, which may limit the fusion performance. Furthermore, normalized coefficient values are employed to stimulate the PCNN, which may cause detail loss and blurring effect in the fused image. Electrophysiological experiments have proved that the neuron representations of complex stimulation in the cat visual cortex are represented by sparse coding [19–21]. Morphological component analysis (MCA) has been widely studied as effective image decomposition. Combining MCA with SR can acquire the SR of cartoon and texture components of an image [22, 23]. To resolve the disadvantage produced by patch coding, convolutional sparse representation (CSR) has been shown to be more effective than sparse representation in extracting features [24]. It is implemented on the whole image instead of a local image patch. Based on the above considerations, this paper presents a medical image fusion algorithm using convolutional sparsity to stimulate the PCNN based on morphological component analysis (MCA-CS-PCNN). Source images are first decomposed into cartoon components and texture components by MCA, and CSR of cartoon layers and texture layers is obtained by prelearned dictionaries. Then, convolutional sparsity is employed to stimulate the PCNN for processing cartoon layers and texture layers. The MIF image is computed via combining fused cartoon layers and texture layers. We test the performance of the proposed MCA-CS-PCNN fusion method, and the experimental results verify the advantages of our fusion strategy.


## 2. Related Work

### 2.1. Convolutional Sparsity Based on Morphological Component Analysis (CSMCA)

Convolutional sparsity is a sparse representation model applying the convolutional form [24], which is based on an entire image rather than overlapped patch. The CSR is defined as(1)argminsn12I−∑n=1Ndn×sn22+γ∑n=1nsn1,where I denotes an image, sn and dn denote the global sparse coefficient maps and dictionary filter, respectively, ∗ represents the convolution operator, and γ is the regularization parameter.Morphological component analysis of an image is regarded as a linear combination of different components, which is defined as [23](2)I=Ic+It,whereIc and It denote cartoon components and texture components, respectively. According to CSR theory, the model of convolutional sparsity based on morphological component analysis (CSMCA) is expressed as(3)argmincc,n,ct,n12I−∑n=1ncbc,v∗sc,n−∑n=1ntbt,n∗st,n22+γc∑n=1ncsc,n1+γc∑n=1ntst,n1,where bc,nn=1nc and sc,nn=1nc denote the dictionary and convolution sparse coefficient corresponding to Ic, respectively. bt,nn=1nt and st,nn=1nc represent the dictionary and convolution sparse coefficient corresponding to It, respectively. The image is computed and denoted by(4)I=∑n=1ncbc,n∗sc,n+∑n=1ntbt,n∗st,n.


### 2.2. Pulse-Coupled Neural Network

The diagrammatic diagram of the simplified PCNN is shown in Figure1. There are three modules in the simplified PCNN model [12], which include the dendritic, linking modulation, and pulse generator, where feeding and linking input are built into the dendritic, denoted by Ex,y=m and Lx,y=m. Fx,y=m and Px,y=m denote the linking modulation and the pulse generator, respectively. The simplified PCNN model is denoted by(5)Ex,ym=Sx,y,(6)Lx,ym=e−μLLx,ym−1+GL∑u,vWx,y,u,vPx,ym−1,(7)Fx,ym=Ex,ym1+ηLx,ym,(8)Tx,ym=e−βTTx,ym−1+ZTPx,ym−1,(9)Px,ym=1,Fx,ym>Tx,ym,0,Tx,ym>Tx,ym,where x, y denote pixel locations, u, v represent the dislocation in the symmetric neighborhood around a pixel, W and Si,jn denote the synaptic weight matrices and the external stimulus, respectively, GL and μL are normalizing constants, and η varies the weight of the linking field, which denotes the linking parameter. The threshold magnitude coefficient and attenuation coefficient are represented by ZT and βT, respectively.Figure 1
Diagrammatic diagram of the simplified PCNN.


## 2.1. Convolutional Sparsity Based on Morphological Component Analysis (CSMCA)

Convolutional sparsity is a sparse representation model applying the convolutional form [24], which is based on an entire image rather than overlapped patch. The CSR is defined as(1)argminsn12I−∑n=1Ndn×sn22+γ∑n=1nsn1,where I denotes an image, sn and dn denote the global sparse coefficient maps and dictionary filter, respectively, ∗ represents the convolution operator, and γ is the regularization parameter.Morphological component analysis of an image is regarded as a linear combination of different components, which is defined as [23](2)I=Ic+It,whereIc and It denote cartoon components and texture components, respectively. According to CSR theory, the model of convolutional sparsity based on morphological component analysis (CSMCA) is expressed as(3)argmincc,n,ct,n12I−∑n=1ncbc,v∗sc,n−∑n=1ntbt,n∗st,n22+γc∑n=1ncsc,n1+γc∑n=1ntst,n1,where bc,nn=1nc and sc,nn=1nc denote the dictionary and convolution sparse coefficient corresponding to Ic, respectively. bt,nn=1nt and st,nn=1nc represent the dictionary and convolution sparse coefficient corresponding to It, respectively. The image is computed and denoted by(4)I=∑n=1ncbc,n∗sc,n+∑n=1ntbt,n∗st,n.


## 2.2. Pulse-Coupled Neural Network

The diagrammatic diagram of the simplified PCNN is shown in Figure1. There are three modules in the simplified PCNN model [12], which include the dendritic, linking modulation, and pulse generator, where feeding and linking input are built into the dendritic, denoted by Ex,y=m and Lx,y=m. Fx,y=m and Px,y=m denote the linking modulation and the pulse generator, respectively. The simplified PCNN model is denoted by(5)Ex,ym=Sx,y,(6)Lx,ym=e−μLLx,ym−1+GL∑u,vWx,y,u,vPx,ym−1,(7)Fx,ym=Ex,ym1+ηLx,ym,(8)Tx,ym=e−βTTx,ym−1+ZTPx,ym−1,(9)Px,ym=1,Fx,ym>Tx,ym,0,Tx,ym>Tx,ym,where x, y denote pixel locations, u, v represent the dislocation in the symmetric neighborhood around a pixel, W and Si,jn denote the synaptic weight matrices and the external stimulus, respectively, GL and μL are normalizing constants, and η varies the weight of the linking field, which denotes the linking parameter. The threshold magnitude coefficient and attenuation coefficient are represented by ZT and βT, respectively.Figure 1
Diagrammatic diagram of the simplified PCNN.


## 3. Proposed MIF Fusion Framework

### 3.1. MCA-CS-PCNN

The flowchart of the MCA-CS-PCNN framework is shown in Figure2. Images A and B denote different source images, which are decomposed into cartoon components AC,BC and the texture components AT,BT by applying MCA, respectively. According to equations (1)–(4), the CSR of cartoon components and texture components is computed as(10)cc,nAn=1nc,ct,nAn=1nt=ACSMCAC,ACSMCAT=CSMCAA,cc,nBn=1nc,ct,nBn=1nt=BCSMCAC,BCSMCAT=CSMCAB,where CSMCA⋅ represents CSMCA functions, described in Section 2.1, cc,nAn=1nc and ct,nAn=1nt denote the convolution sparse coefficient map of AC,AT, respectively, and cc,nBn=1nc and ct,nBn=1nt denote the convolution sparse coefficient map of BC,BT.Figure 2
Flowchart of the anatomical and anatomical image fusion method.Next, the convolutional sparse representation is used to stimulate the PCNN because complex stimulation in the cat visual cortex is based on sparse coding.ACSMCAC,BCSMCAC and ACSMCAT,BCSMCAT are employed to stimulate the PCNN for processing AC,BC and AT,BT, respectively.(11)TAC=PCNNACMCAC,TBC=PCNNBCSMCAC,TAT=PCNNACSMCAT,TBT=PCNNBCSMCAT,where PCNN⋅ denotes the PCNN functions; the firing time matrices TAC,TBC of AC,BC and TAT,TBT of AT,BT are obtained according to equations (5)–(9), until the iteration number m=Mmax, where Mmax denotes the max iteration times, and then iteration stops.Then, fused coefficients of the convolution sparse coefficient map in cartoon components are computed by(12)cc,nx,y=maxcc,nAx,y,cc,nBx,y,TAC=TBC,cc,nAx,y,TAC>TBC,cc,nBx,y,TAC<TBC.The fused coefficients of the convolution sparse coefficient map in texture components are computed by(13)ct,nx,y=maxct,nAx,y,ct,nBx,y,TAT=TBT,ct,nAx,y,TAT>TBT,ct,nBx,y,TAT<TBT.The fused image of cartoon componentFc and fused image of texture component Ft are computed and denoted by(14)Fc=∑n=1ncbc,n∗cc,n,Ft=∑n=1ntbt,n∗ct,n,where bc,nn=1nc and bt,nn=1nt are dictionaries.Finally, the medical fused image is acquired and denoted as(15)F=Fc+FT.


### 3.2. Extension to Anatomical and Functional Image Fusion Based on MCA-CS-PCNN

The proposed MCA-CS-PCNN is extended to conduct anatomical and functional image fusion. Considering that functional images are pseudo-color images, the YUV color space transform has shown to be effective in processing pseudo-color images [10, 16]. Specifically speaking, a functional image with RGB is firstly transformed into the Y channel, U channel, and V channel. Then, the new Y channel is produced via the fusion of the Y channel and grayscale image based on MCA-CS-PCNN, and the new YUV is acquired via merging the new Y, U, and V. Finally, YUV is converted into RGB, and the medical fused image with color is obtained. The flowchart of the anatomical and functional image fusion strategy based on MCA-CS-PCNN is shown in Figure 3.Figure 3
Flowchart of the anatomical and functional image fusion method.


## 3.1. MCA-CS-PCNN

The flowchart of the MCA-CS-PCNN framework is shown in Figure2. Images A and B denote different source images, which are decomposed into cartoon components AC,BC and the texture components AT,BT by applying MCA, respectively. According to equations (1)–(4), the CSR of cartoon components and texture components is computed as(10)cc,nAn=1nc,ct,nAn=1nt=ACSMCAC,ACSMCAT=CSMCAA,cc,nBn=1nc,ct,nBn=1nt=BCSMCAC,BCSMCAT=CSMCAB,where CSMCA⋅ represents CSMCA functions, described in Section 2.1, cc,nAn=1nc and ct,nAn=1nt denote the convolution sparse coefficient map of AC,AT, respectively, and cc,nBn=1nc and ct,nBn=1nt denote the convolution sparse coefficient map of BC,BT.Figure 2
Flowchart of the anatomical and anatomical image fusion method.Next, the convolutional sparse representation is used to stimulate the PCNN because complex stimulation in the cat visual cortex is based on sparse coding.ACSMCAC,BCSMCAC and ACSMCAT,BCSMCAT are employed to stimulate the PCNN for processing AC,BC and AT,BT, respectively.(11)TAC=PCNNACMCAC,TBC=PCNNBCSMCAC,TAT=PCNNACSMCAT,TBT=PCNNBCSMCAT,where PCNN⋅ denotes the PCNN functions; the firing time matrices TAC,TBC of AC,BC and TAT,TBT of AT,BT are obtained according to equations (5)–(9), until the iteration number m=Mmax, where Mmax denotes the max iteration times, and then iteration stops.Then, fused coefficients of the convolution sparse coefficient map in cartoon components are computed by(12)cc,nx,y=maxcc,nAx,y,cc,nBx,y,TAC=TBC,cc,nAx,y,TAC>TBC,cc,nBx,y,TAC<TBC.The fused coefficients of the convolution sparse coefficient map in texture components are computed by(13)ct,nx,y=maxct,nAx,y,ct,nBx,y,TAT=TBT,ct,nAx,y,TAT>TBT,ct,nBx,y,TAT<TBT.The fused image of cartoon componentFc and fused image of texture component Ft are computed and denoted by(14)Fc=∑n=1ncbc,n∗cc,n,Ft=∑n=1ntbt,n∗ct,n,where bc,nn=1nc and bt,nn=1nt are dictionaries.Finally, the medical fused image is acquired and denoted as(15)F=Fc+FT.


## 3.2. Extension to Anatomical and Functional Image Fusion Based on MCA-CS-PCNN

The proposed MCA-CS-PCNN is extended to conduct anatomical and functional image fusion. Considering that functional images are pseudo-color images, the YUV color space transform has shown to be effective in processing pseudo-color images [10, 16]. Specifically speaking, a functional image with RGB is firstly transformed into the Y channel, U channel, and V channel. Then, the new Y channel is produced via the fusion of the Y channel and grayscale image based on MCA-CS-PCNN, and the new YUV is acquired via merging the new Y, U, and V. Finally, YUV is converted into RGB, and the medical fused image with color is obtained. The flowchart of the anatomical and functional image fusion strategy based on MCA-CS-PCNN is shown in Figure 3.Figure 3
Flowchart of the anatomical and functional image fusion method.


## 4. Experiments

### 4.1. Experimental Settings

To test and verify the performance of the MCA-CS-PCNN fusion algorithm, ten pairs of medical images with the same size of 256 × 256 pixels are used to conduct the experiments, including five pairs of anatomical image and functional image fusion and five pairs of anatomical image and anatomical image fusion (Figures4 and 5). Five representative medical image fusion algorithms are selected for experimental comparison; they are convolutional sparse representation (CSR) [24], NSCT-based modified spatial frequency and PCNN (NSCT-MSF-PCNN) [14], guided filtering (GFF) [25], cross-scale coefficient selection (CSCS) [26], and sparse representation based on the Laplacian pyramid (LP-SR) [11]. Objective quality evaluation is important for image quality [27–31]. The existing fusion quality metrics include the human perception quality metric QHP [32], feature mutual information quality metric QFMI [33], spatial frequency quality metric QSF [34], standard deviation quality metric QSD [11], nonlinear correlation information entropy metric QNCIE [35], and mutual information metric QMI [36]. In the above quality metrics, the higher the values of QHP, QFMI, QSF, QSD, QMI, and QNCIE, the higher the fusion performance.Figure 4
Five pairs of source images with anatomical and anatomical.
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)Figure 5
Five pairs of source images with anatomical and functional.
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)


### 4.2. Analysis of Experimental Results

In the example of anatomical and anatomical image fusion, we can see that the anatomical information of the bones or soft tissues is contained in the fused images by the six algorithms; still, differences between fused images can be clearly distinguished, such as focal regions blur (Figures6(a), 6(c), and 6(d)), information of soft tissues regions are missing (Figures 6(b) and 6(e)). Our method obtains better performance than other methods. The example of anatomical and functional image fusion shows that the fused images obtained by GFF and CSCS lead to the loss of color information (Figures 7(a), 7(c), and 7(d)), and the NSCT-MSF-PCNN and LP-SR algorithms lead to poor visual effect, for instance, the details of the anatomical image are lost (Figures 7(b) and 7(e)). From the comparisons, our proposed algorithm demonstrates more advantages than the existing algorithms.Figure 6
Five anatomical and anatomical image fusion results obtained by six methods. (a) CSR. (b) NSCT-MSF-PCNN. (c) GFF. (d) CSCS. (e) LP-SR. (f) Proposed.
(a)(b)(c)(d)(e)(f)Figure 7
Five anatomical and functional image fusion results obtained by six methods. (a) CSR. (b) NSCT-MSF-PCNN. (c) GFF. (d) CSCS. (e) LP-SR. (f) Proposed.
(a)(b)(c)(d)(e)(f)Tables1 and 2 give the objective evaluation results of the proposed MCA-CS-PCNN fusion algorithm and five fusion methods via using objective fusion quality metrics. We mark best results employing the boldface in each row. Table 1 shows an objective evaluation of the fused image about anatomical image and functional image. We QSF values are only slightly lower than LP-SR in the second pair of images of Figure 4. Our method achieves the significant superiority. From Table 2, it can be see that QMI values are only slightly lower than GFF in the second pair of images of Figure 5, and the values of QFMI and QSD in our proposed algorithm demonstrate advantages.Table 1
Objective evaluation for anatomical and anatomical image fusion.
MetricCSRNSCT-MSF-PCNNGFFCSCSLP-SRProposedQHP192.5371217.4689204.7978191.4941217.2586475.4320QFMI0.71660.48740.77470.65420.76610.9024QSF6.55016.98216.83116.70447.01357.6950QSD60.862360.898861.333859.860162.092870.2132QHP877.7084962.9514849.00391033.4054975.27391225.7900QFMI0.70680.53800.72380.63720.70900.8649QSF6.12206.52936.38585.67666.90766.8722QSD45.448648.448749.147543.102149.658851.1843QHP306.5058314.8874361.3098424.6304276.2802580.2270QFMI0.76000.52940.78650.67270.77500.8750QSF6.04096.40686.51225.98816.62946.8274QSD58.790258.195958.475556.845859.701169.6467QHP267.9495317.9863222.8093467.2239253.2109481.7950QFMI0.74430.49250.77470.65560.76510.8862QSF6.40276.86046.89626.40127.02557.1951QSD61.173867.029665.278359.989067.779178.0285QHP90.536387.739479.0995195.928294.2120220.7360QFMI0.74670.57540.80300.70830.77410.8719QSF5.86306.20275.95475.99076.38366.8953QSD55.833756.424157.161554.756356.657360.1770Table 2
Objective evaluation for anatomical and functional image fusion.
MetricCSRNSCT-MSF-PCNNGFFCSCSLP-SRProposedQNCIE0.80650.80550.80640.80670.80600.8071QMI0.66230.51710.64280.65970.61780.7081QFMI0.65600.46910.66210.56000.65910.8489QSD48.909553.846950.027847.376154.448555.5928QNCIE0.80580.80510.81670.80580.80560.8277QMI0.67600.53731.22420.65030.66400.8370QFMI0.72650.46840.78940.64370.71300.8674QSD59.862771.660256.729057.527373.709683.8645QNCIE0.80640.80530.80630.80670.80610.8071QMI0.64190.49940.62220.64170.60530.6914QFMI0.64590.46830.65920.53760.64760.8457QSD48.464356.570750.255946.497955.279656.1166QNCIE0.80600.80520.81580.80590.80560.8072QMI0.70600.55671.25650.66080.66600.8220QFMI0.73310.49110.79670.63450.72290.8275QSD59.443275.104359.487255.598271.437689.1413QNCIE0.80390.80300.80550.80490.80350.8062QMI0.56980.36520.67210.62710.50770.8241QFMI0.73800.52540.75100.66570.74310.8608QSD35.390346.079749.600044.312445.595251.4487


## 4.1. Experimental Settings

To test and verify the performance of the MCA-CS-PCNN fusion algorithm, ten pairs of medical images with the same size of 256 × 256 pixels are used to conduct the experiments, including five pairs of anatomical image and functional image fusion and five pairs of anatomical image and anatomical image fusion (Figures4 and 5). Five representative medical image fusion algorithms are selected for experimental comparison; they are convolutional sparse representation (CSR) [24], NSCT-based modified spatial frequency and PCNN (NSCT-MSF-PCNN) [14], guided filtering (GFF) [25], cross-scale coefficient selection (CSCS) [26], and sparse representation based on the Laplacian pyramid (LP-SR) [11]. Objective quality evaluation is important for image quality [27–31]. The existing fusion quality metrics include the human perception quality metric QHP [32], feature mutual information quality metric QFMI [33], spatial frequency quality metric QSF [34], standard deviation quality metric QSD [11], nonlinear correlation information entropy metric QNCIE [35], and mutual information metric QMI [36]. In the above quality metrics, the higher the values of QHP, QFMI, QSF, QSD, QMI, and QNCIE, the higher the fusion performance.Figure 4
Five pairs of source images with anatomical and anatomical.
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)Figure 5
Five pairs of source images with anatomical and functional.
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)


## 4.2. Analysis of Experimental Results

In the example of anatomical and anatomical image fusion, we can see that the anatomical information of the bones or soft tissues is contained in the fused images by the six algorithms; still, differences between fused images can be clearly distinguished, such as focal regions blur (Figures6(a), 6(c), and 6(d)), information of soft tissues regions are missing (Figures 6(b) and 6(e)). Our method obtains better performance than other methods. The example of anatomical and functional image fusion shows that the fused images obtained by GFF and CSCS lead to the loss of color information (Figures 7(a), 7(c), and 7(d)), and the NSCT-MSF-PCNN and LP-SR algorithms lead to poor visual effect, for instance, the details of the anatomical image are lost (Figures 7(b) and 7(e)). From the comparisons, our proposed algorithm demonstrates more advantages than the existing algorithms.Figure 6
Five anatomical and anatomical image fusion results obtained by six methods. (a) CSR. (b) NSCT-MSF-PCNN. (c) GFF. (d) CSCS. (e) LP-SR. (f) Proposed.
(a)(b)(c)(d)(e)(f)Figure 7
Five anatomical and functional image fusion results obtained by six methods. (a) CSR. (b) NSCT-MSF-PCNN. (c) GFF. (d) CSCS. (e) LP-SR. (f) Proposed.
(a)(b)(c)(d)(e)(f)Tables1 and 2 give the objective evaluation results of the proposed MCA-CS-PCNN fusion algorithm and five fusion methods via using objective fusion quality metrics. We mark best results employing the boldface in each row. Table 1 shows an objective evaluation of the fused image about anatomical image and functional image. We QSF values are only slightly lower than LP-SR in the second pair of images of Figure 4. Our method achieves the significant superiority. From Table 2, it can be see that QMI values are only slightly lower than GFF in the second pair of images of Figure 5, and the values of QFMI and QSD in our proposed algorithm demonstrate advantages.Table 1
Objective evaluation for anatomical and anatomical image fusion.
MetricCSRNSCT-MSF-PCNNGFFCSCSLP-SRProposedQHP192.5371217.4689204.7978191.4941217.2586475.4320QFMI0.71660.48740.77470.65420.76610.9024QSF6.55016.98216.83116.70447.01357.6950QSD60.862360.898861.333859.860162.092870.2132QHP877.7084962.9514849.00391033.4054975.27391225.7900QFMI0.70680.53800.72380.63720.70900.8649QSF6.12206.52936.38585.67666.90766.8722QSD45.448648.448749.147543.102149.658851.1843QHP306.5058314.8874361.3098424.6304276.2802580.2270QFMI0.76000.52940.78650.67270.77500.8750QSF6.04096.40686.51225.98816.62946.8274QSD58.790258.195958.475556.845859.701169.6467QHP267.9495317.9863222.8093467.2239253.2109481.7950QFMI0.74430.49250.77470.65560.76510.8862QSF6.40276.86046.89626.40127.02557.1951QSD61.173867.029665.278359.989067.779178.0285QHP90.536387.739479.0995195.928294.2120220.7360QFMI0.74670.57540.80300.70830.77410.8719QSF5.86306.20275.95475.99076.38366.8953QSD55.833756.424157.161554.756356.657360.1770Table 2
Objective evaluation for anatomical and functional image fusion.
MetricCSRNSCT-MSF-PCNNGFFCSCSLP-SRProposedQNCIE0.80650.80550.80640.80670.80600.8071QMI0.66230.51710.64280.65970.61780.7081QFMI0.65600.46910.66210.56000.65910.8489QSD48.909553.846950.027847.376154.448555.5928QNCIE0.80580.80510.81670.80580.80560.8277QMI0.67600.53731.22420.65030.66400.8370QFMI0.72650.46840.78940.64370.71300.8674QSD59.862771.660256.729057.527373.709683.8645QNCIE0.80640.80530.80630.80670.80610.8071QMI0.64190.49940.62220.64170.60530.6914QFMI0.64590.46830.65920.53760.64760.8457QSD48.464356.570750.255946.497955.279656.1166QNCIE0.80600.80520.81580.80590.80560.8072QMI0.70600.55671.25650.66080.66600.8220QFMI0.73310.49110.79670.63450.72290.8275QSD59.443275.104359.487255.598271.437689.1413QNCIE0.80390.80300.80550.80490.80350.8062QMI0.56980.36520.67210.62710.50770.8241QFMI0.73800.52540.75100.66570.74310.8608QSD35.390346.079749.600044.312445.595251.4487


## 5. Conclusion

This paper proposes a perceptual medical image fusion framework based on morphological component analysis combining convolutional sparsity and pulse-coupled neural network, which is called MCA-CS-PCNN for short. It is basically based on the visual system feature that the cat visual cortex can produce complex stimulation, and the neuron representations of complex stimulation can be represented using sparse coding. To this end, we first decomposed source images into cartoon components and texture components by morphological component analysis, and convolutional sparse representation of cartoon layers and texture layers is obtained by prelearned dictionaries. Then, convolutional sparsity is employed to stimulate the PCNN for processing cartoon layers and texture layers. Finally, the medical fused image is computed via combining fused cartoon layers and texture layers. The experimental results verify that the proposed model can produce high performance, which is superior to the state-of-the-art fusion strategy.



---

*Source: 6647200-2021-03-29.xml*