# Medical Image Fusion Algorithm Based on Nonlinear Approximation of Contourlet Transform and Regional Features

Authors: Hui Huang; Xi’an Feng; Jionghui Jiang
Journal: Journal of Electrical and Computer Engineering
 (2017)

---

## Abstract

According to the pros and cons of contourlet transform and multimodality medical imaging, here we propose a novel image fusion algorithm that combines nonlinear approximation of contourlet transform with image regional features. The most important coefficient bands of the contourlet sparse matrix are retained by nonlinear approximation. Low-frequency and high-frequency regional features are also elaborated to fuse medical images. The results strongly suggested that the proposed algorithm could improve the visual effects of medical image fusion and image quality, image denoising, and enhancement.


---

## Body

## 1. Introduction

As an analytical probe, there has been an ever-increasing interest in developing and applying medical imaging technique to problems in clinical diagnosis. The list of possible applications of X-ray, ultrasound, CT, MRI, SPECT, and PET in clinical diagnosis continues to grow and diversify. Although those imaging technologies have given clinicians an unprecedented toolbox to aid in clinical decision-making, advances in image fusion of comprehensive morphological and functional information retrieved from different imaging technologies could enable physicians to identify human anatomy, physiology, and pathology as well as diseases at an even earlier stage.Recently, there has been active research using medical image fusion technology for clinical applications in the academic community. Many researchers have proposed various image fusion algorithms which have achieved good results, such as the Laplacian pyramid transform [1], the contrast pyramid transform [2, 3] techniques based on the morphological pyramid transform [4], techniques based on the pyramid gradient transform [5], wavelet transform [6–8], Ridgelet, and Curvelet. DO and Martin Vetterli proposed the contourlet transform in 2002, which is a “real” two-dimensional image representation based on wavelet multiscale analysis and known as a Pyramidal Directional Filter Bank (PDFB). The multiscale geometric analytical tool used in contourlet transform demonstrates excellent spatial and frequency domain localization properties of wavelet analysis, as well as the bonus of multidirectional, multiscale characteristics and good anisotropy, demonstrating its suitability to describe the geometric characteristics of an image [9, 10]. Additionally, the contourlet wavelet transform with smaller contourlet coefficients is able to express abundant sparse matrices with depicting image edges, such as curves, straight lines, and other features. After contourlet transformation, the image becomes more focused, which is conducive to the tracking and analysis of important image features. Contourlet transform can decompose in any direction and on any scale, which is key to the accurate description of image contours and directional textural information. To date, many scholars have applied the contourlet transform to image fusion and reported good results, particularly when combined with the image characteristics of contourlet transform fusion [11, 12], nonsubsampled contourlet image fusion [13, 14], and nonlinear approximation of contourlet transform fusion [15, 16].This paper proposes an image fusion algorithm based on the analysis of a large number of image fusion contourlet transform techniques, which combines nonlinear approximation of contourlet transform and regional image features. First, contourlet decomposition is employed to extract the high-frequency and low-frequency regions of an image. The low-frequency coefficients are retained, and nonlinear approximations of the most significant high-frequency coefficients are obtained. Then, the low-frequency coefficients and the most significant high-frequency coefficients are combined via image fusion. The coefficient matrix module is used to calculate the energy region near the center point of the low-frequency region, and a reasonable fusion coefficient is chosen. Based on the most significant high-frequency coefficients, this paper employs the fusion rule of salience/match measure with a threshold. CT and MRI image simulations and experimental results indicate that the proposed method achieves better fusion performance and visual effects than the wavelet transform and traditional contourlet fusion methods.


## 2. Nonlinear Approximation of Contourlet Transform Algorithm

A contourlet transform model of the filter group can be extended into a continuous function of square spaceL
2
R
2 [17, 18]. In a continuous domain contourlet transform, L
2
(
R
2
) is decomposed into multiscale, multidirectional subspace sequence by the use of an iterative filter group, as shown the following equation:(1)
L
2
R
2
=
V
j
0
⊕
⨁
j
≤
j
0
⊕
k
=
0
2
l
j
W
j
,
k
l
j
.The definition of spaceV
j
0 and W
j is consistent with wavelet decomposition [19]. V
j
0 is the approximate space. The scaling function ϕ provides an approximation component on the 2
J scale represented by ϕ
J
,
n
(
t
)
=
ϕ
J
(
t
-
2
j
n
)
n
∈
z
2, as generated by zooming and panning the scaling function orthogonal basis. W
j is decomposed into 2
j directional subspace W
j
,
k
l
j, expressed as W
j
=
⨁
k
=
0
2
l
j
-
1
W
j
,
k
(
l
j
). The W
j
,
k
(
l
j
) space is defined in the rectangular frame 2
j
+
l
j
-
2
×
2
j or 2
j
×
2
j
+
l
j
-
2, which belongs to ρ
j
,
k
,
n
l
j
t
n
∈
Z
2, as shown in the following equation: (2)
ρ
j
,
k
,
n
l
j
t
=
∑
m
∈
Z
2
g
k
l
j
m
-
S
k
l
j
n
u
j
,
m
t
.In Formula (2), g
k
l
j is a low-pass analysis filter of PDB. The sampling matrix S
k
(
l
j
) can be expressed as follows:(3)
S
k
l
j
=
diag
⁡
2
l
j
-
1
,
2
0
≤
k
<
2
l
j
-
1
diag
⁡
2
,
2
l
j
-
1
2
l
j
-
1
≤
k
<
2
l
j
.In Formula (3), the k parameters directly determine the conversion of orientation, that is, the horizontal or vertical bias. According to multiresolution analysis theory, ρ
j
,
k
,
n
l
j
(
t
)
n
∈
Z
2 can be obtained from the original function and its translation, as shown in the following equation:(4)
ρ
j
,
k
,
n
l
j
t
=
ρ
j
,
k
l
j
t
-
2
j
-
1
S
k
l
j
n
.According to the theory described above,ρ
j
,
k
,
n
l
j
t is a continuous domain contourlet, while j, k, and n represent the scale, orientation, and position parameters of the contourlet, respectively.Given a set of base functionsϕ
n
n
=
1
∞ in a nonlinear approximation of contourlet transform, the function can be expanded to f
=
∑
n
=
1
∞
c
n
ϕ
n. Then the maximum absolute values of M
c
n coefficients are used to approximate the original function, expressed as f
~
=
∑
n
∈
I
M
c
n
ϕ
n, where I
M is the index of the maximum absolute value of the coefficient M and f
~ represents the nonlinear approximation of the function [20].


## 3. Feature Matching Algorithm

### 3.1. Low-Frequency Fusion Algorithm

The low-frequency subband retains the profile information of the original image; the low-frequency region is processed as much as possible in order to retain the profile characteristics. In this paper, the window coefficient matrix is employed to calculate the energy in the region near the image center point, which not only takes regional factors into account but also retains the characteristics of directivity and highlights the central pixel. The energyE of the low-frequency region can be defined as follows:(5)
E
j
0
m
,
n
=
x
,
y
∈
Ω
m
,
n
:
z
,
V
j
0
m
,
n
.Ω
m
,
n represents the center of the neighborhood of (
m
,
n
), x
,
y represents the center of the proximity panel of m
,
n, z is the area coefficient matrix, and V
j
0 is the low-frequency subband coefficient of the fused image. According to the rule of the low-frequency region of energy fusion, the energy E of the low-frequency region must be calculated first. X is the center of the two adjacent subimages of the low-frequency region and the center region of the two adjacent low-frequency subimages of (
m
,
n
). Then, the absolute value of the regional energy E can be expressed as follows:(6)
V
j
0
m
,
n
=
V
j
0
A
m
,
n
E
J
0
A
m
,
n
>
E
J
0
B
m
,
n
V
j
0
B
m
,
n
E
J
0
A
m
,
n
≤
E
J
0
B
m
,
n
.


### 3.2. High-Frequency Fusion Algorithm

This paper utilizes high-frequency coefficients and the rules of the match measure with a threshold. The local energy of the high-frequency region [21] can be defined as follows:(7)
S
i
,
j
,
k
=
∑
m
∑
n
C
i
+
m
,
j
+
n
,
k
2
.C
i
,
j
,
k represents the position of the high-frequency coefficients of i
,
j on the k decomposition level. The size of the image in a neighborhood of the window (typically 3 × 3 or 5 × 5 pixels) is defined by m
,
n. The neighborhood of the sum of the square of the high-frequency coefficients is represented by the local energy of point i
,
j
,
k. The match degree of point i
,
j
,
k is defined as follows:(8)
M
A
B
i
,
j
,
k
=
2
∑
m
∑
n
C
A
i
+
m
,
j
+
n
,
k
C
B
i
+
m
,
j
+
n
,
k
S
A
i
,
j
,
k
S
B
i
,
j
,
k
.The fusion rule of the high-frequency coefficients is defined by the following formula:(9)
C
i
,
j
,
k
=
W
A
C
A
i
,
j
,
k
+
W
B
C
B
i
,
j
,
k
.Match degree, which is measured by the matching degree of the feature information in corresponding position of the two original imagesA andB, determines the proportion of the characteristic information of different original images. The point i
,
j
,
k’s match degree is determined by the rules as follows:(1)
IfM
A
B
≤
T, then(10a)
W
A
=
1
,
W
B
=
0
,
S
A
>
S
B
,
W
A
=
0
,
W
B
=
1
,
S
A
≤
S
B
.
(2)
IfM
A
B
>
T, then(10b)
W
A
=
1
-
W
B
,
W
B
=
1
2
-
1
-
M
A
B
2
1
-
T
,
S
A
>
S
B
,
W
A
=
1
2
-
1
-
M
A
B
2
1
-
T
,
W
B
=
1
-
W
A
,
S
A
≤
S
B
.In the rules listed above,M
A
B represents the match degree, and T
∈
(
0,1
) represents the matching threshold. When M
A
B
≤
T, we take the larger value of the local energy C
A
i
,
j
,
k and C
B
i
,
j
,
k as high-frequency coefficient. When M
A
B
>
T, we take the value W
A
C
A
i
,
j
,
k
+
W
B
C
B
i
,
j
,
k, where the weight W
A and W
B have correlation with the degree of matching, and W
A
+
W
B
=
1. Obviously, the calculation process of feature matching rules demonstrates good locality, because the fusion results of the high-frequency coefficient value at i
,
j
,
k are only determined by the coefficient values which are contained by the m
×
n neighborhood of point i
,
j
,
k.


## 3.1. Low-Frequency Fusion Algorithm

The low-frequency subband retains the profile information of the original image; the low-frequency region is processed as much as possible in order to retain the profile characteristics. In this paper, the window coefficient matrix is employed to calculate the energy in the region near the image center point, which not only takes regional factors into account but also retains the characteristics of directivity and highlights the central pixel. The energyE of the low-frequency region can be defined as follows:(5)
E
j
0
m
,
n
=
x
,
y
∈
Ω
m
,
n
:
z
,
V
j
0
m
,
n
.Ω
m
,
n represents the center of the neighborhood of (
m
,
n
), x
,
y represents the center of the proximity panel of m
,
n, z is the area coefficient matrix, and V
j
0 is the low-frequency subband coefficient of the fused image. According to the rule of the low-frequency region of energy fusion, the energy E of the low-frequency region must be calculated first. X is the center of the two adjacent subimages of the low-frequency region and the center region of the two adjacent low-frequency subimages of (
m
,
n
). Then, the absolute value of the regional energy E can be expressed as follows:(6)
V
j
0
m
,
n
=
V
j
0
A
m
,
n
E
J
0
A
m
,
n
>
E
J
0
B
m
,
n
V
j
0
B
m
,
n
E
J
0
A
m
,
n
≤
E
J
0
B
m
,
n
.


## 3.2. High-Frequency Fusion Algorithm

This paper utilizes high-frequency coefficients and the rules of the match measure with a threshold. The local energy of the high-frequency region [21] can be defined as follows:(7)
S
i
,
j
,
k
=
∑
m
∑
n
C
i
+
m
,
j
+
n
,
k
2
.C
i
,
j
,
k represents the position of the high-frequency coefficients of i
,
j on the k decomposition level. The size of the image in a neighborhood of the window (typically 3 × 3 or 5 × 5 pixels) is defined by m
,
n. The neighborhood of the sum of the square of the high-frequency coefficients is represented by the local energy of point i
,
j
,
k. The match degree of point i
,
j
,
k is defined as follows:(8)
M
A
B
i
,
j
,
k
=
2
∑
m
∑
n
C
A
i
+
m
,
j
+
n
,
k
C
B
i
+
m
,
j
+
n
,
k
S
A
i
,
j
,
k
S
B
i
,
j
,
k
.The fusion rule of the high-frequency coefficients is defined by the following formula:(9)
C
i
,
j
,
k
=
W
A
C
A
i
,
j
,
k
+
W
B
C
B
i
,
j
,
k
.Match degree, which is measured by the matching degree of the feature information in corresponding position of the two original imagesA andB, determines the proportion of the characteristic information of different original images. The point i
,
j
,
k’s match degree is determined by the rules as follows:(1)
IfM
A
B
≤
T, then(10a)
W
A
=
1
,
W
B
=
0
,
S
A
>
S
B
,
W
A
=
0
,
W
B
=
1
,
S
A
≤
S
B
.
(2)
IfM
A
B
>
T, then(10b)
W
A
=
1
-
W
B
,
W
B
=
1
2
-
1
-
M
A
B
2
1
-
T
,
S
A
>
S
B
,
W
A
=
1
2
-
1
-
M
A
B
2
1
-
T
,
W
B
=
1
-
W
A
,
S
A
≤
S
B
.In the rules listed above,M
A
B represents the match degree, and T
∈
(
0,1
) represents the matching threshold. When M
A
B
≤
T, we take the larger value of the local energy C
A
i
,
j
,
k and C
B
i
,
j
,
k as high-frequency coefficient. When M
A
B
>
T, we take the value W
A
C
A
i
,
j
,
k
+
W
B
C
B
i
,
j
,
k, where the weight W
A and W
B have correlation with the degree of matching, and W
A
+
W
B
=
1. Obviously, the calculation process of feature matching rules demonstrates good locality, because the fusion results of the high-frequency coefficient value at i
,
j
,
k are only determined by the coefficient values which are contained by the m
×
n neighborhood of point i
,
j
,
k.


## 4. Experimental Results and Analysis

The CT and MRI image fusion experimental simulations were implemented on PIV 2.4 GHz, using a 4 GB RAM PC as the development platform of Matlab7.0. After nonlinear approximation contourlet transform was performed, a 3 × 3 feature region is calculated in the high-frequency and low-frequency regions, and the high-frequency region match threshold is 0.75 as proposed by Burt and Lolczynski [21].Figure1 depicts the results of nonlinear approximation contourlet transform performed on an MRI image. The various proportions of nonlinear approximation retain the most significant coefficients at high-frequency subbands. Table 1 describes the MRI image feature coefficients and the most significant coefficients at various approximation proportions.Table 1
Significant coefficients.
MRI image
10% coefficients
30% coefficients
50% coefficients
Significant coefficients
86016
6554
19661
32768Figure 1
Nonlinear approximation of contourlet.
(a)
MRI image
(b)
10% coefficients
(c)
30% coefficients
(d)
50% coefficientsFigure2 depicts the matching fusion of MRI and CT image characteristics based on nonlinear approximation contourlet transform. A constant low-frequency region is maintained after nonlinear approximation of significant coefficients at high-frequency subbands, and the images are then fused according to their regional characteristics. This paper quantitatively analyzes the effects of fusion based on the indicators of standard deviation, correlation coefficients, entropy, and mutual information [22, 23].Figure 2
Fusion of contourlet nonlinear approximation.
(a)
CT image
(b)
MRI image
(c)
10% coefficients
(d)
30% coefficients
(e)
50% coefficients
(f)
70% coefficients
(g)
90% coefficients


### 4.1. Standard Deviation

The standard deviation (STD) reflects the contrast change of an image: the larger the value, the clearer the edge contour; the definition of STD is given as follows:(11)
S
T
D
=
∑
i
=
1
M
∑
j
=
1
N
f
i
,
j
-
f
^
2
M
×
N
.In the formula,f
i
,
j represents the result fusion image, while M
×
N represents the image f, and f
^ represents the mean value of f.


### 4.2. Correlation Coefficients

Correlation coefficient (CC) is a measure of the similarity of two images: the greater the correlation coefficient, the better the fusion effect; CC is defined as(12)
C
C
X
,
Y
=
∑
i
=
1
M
∑
j
=
1
N
X
i
,
j
-
X
-
Y
i
,
j
-
Y
-
∑
i
=
1
M
∑
j
=
1
N
X
i
,
j
-
X
-
2
∑
i
=
1
M
∑
j
=
1
N
Y
i
,
j
-
Y
-
2
.


### 4.3. Entropy

Entropy (EN) measures the amount of information maintained by the fused image. The larger the entropy value is, the more information the result image has. EN is defined as(13)
E
N
=
-
∑
0
255
P
i
x
log
2
⁡
p
i
x
,in whichP
i
x is defined as the normalized histogram of the variable x and ∑
0
255
P
i
x
=
1.


### 4.4. Mutual Information

Mutual information (MI) can be used to measure the mutual correlation or similarity of the two input images. The higher the mutual information of the fused image, the more information extracted from the original images and the better the fusion effect. For the two input discrete image signals, MI is defined as(14)
M
I
x
,
y
=
H
x
+
H
y
-
H
x
,
y
,in whichH
(
x
) is an entropy function; H
(
x
)
=
-
∑
0
255
P
i
x
log
2
⁡
p
i
(
x
), while H
x
,
y is a joint entropy function; H
(
x
,
y
)
=
-
∑
0
255
P
i
x
,
y
log
2
⁡
p
i
(
x
,
y
). P
i
(
x
,
y
) is defined as histogram of normalized distribution of x and  y and ∑
0
255
P
i
x
,
y
=
1.As shown in Table2, the maximum value of the standard deviation in the fused image can be obtained for less than 50% significant coefficients, which demonstrate sharper profile edges and the highest resolution at 50% nonlinear approximation. This may be the result of few significant coefficients which will in turn result in less extraction of edge contours; alternatively, too many significant coefficients will lead to the extraction of excess noise. Correlation coefficients, mutual information, and entropy have little effect on image fusion with various proportions of significant coefficients. The primary reason for this is that the low-frequency subband retains the profile information of the original image, while fused images do not achieve nonlinear approximation in the low-frequency region. Results indicate that fusion image quality demonstrates a nonlinear relationship to the proportion of significant coefficients. When the significant coefficient is 50%, the quality of image fusion is highest.Table 2
Indexes of performance evaluation.
Standard deviation
Correlation coefficients
Entropy
Mutual information
10% significant coefficients
50.48
0.66
7.68
4.51
30% significant coefficients
56.32
0.69
7.68
4.53
50% significant coefficients
63.16
0.70
7.70
4.54
70% significant coefficients
59.32
0.71
7.70
4.55
90% significant coefficients
57.16
0.73
7.72
4.55In order to verify the effectiveness of the algorithm, we compare our result with others from different researchers. Figure3 shows different fusion results for MRI and CT images with different algorithms, in which Figure 3(a) is the experimental result with fusion algorithm based on regional feature of wavelet coefficients [24], Figure 3(b) is experimental result based on Haar wavelet transform [25], Figure 3(c) is the result of coefficient weighted fusion algorithm after contourlet transform [19], Figure 3(d) is the fusion result based on the nonsubsampled contourlet transform (NSCT) and regional features [26], and Figure 3(e) is our experimental fusion result, which is an effect of regional image feature matching after 50% nonlinear estimation to the high-frequency coefficients by the contourlet transform. For the visual assessment, the fusion images (Figures 3(a) and 3(b)) are based on the wavelet transform methods; the color is shallow, and edge profile is not clear enough, which cannot reflect the source CT and MRI images of the details of the information. In Figure 3(c), with contourlet transform, the fusion image effect is slightly better than the wavelet method and can more clearly reflect the content information of the source images. In Figures 3(d) and 3(e), the fusion image not only inherits the bone tissue of the CT image but also can keep the soft tissue of the MRI image; moreover, the image edge detail can be more obvious after image fusion.Figure 3
Fusion results’ comparison.
(a)
CT image
(b)
MRI image
(c)
Wavelet fusion
(d)
Haar fusion
(e)
Contourlet fusion
(f)
Nonsubsampled contourlet fusion
(g)
Nonlinear contourlet fusionAnalyzing the results quantitatively with the indexes shown in Table3, the corresponding STD, CC, EN, and MI of Figures 3(a) and 3(b), which are based on the wavelet transform methods, are relatively small, which shows that the information has been lost in the process of image fusion and also shows a notable difference between the fusion image and the source images. It has been demonstrated that the ability of wavelet transform method to extract information from the source image is poor, and the fusion effect is not good. Figure 3(c) gets more directions in the high-frequency part by using contourlet transform method and can better deal with the direction of the details relative to the wavelet transform, so the coefficient of each index is superior to wavelet fusion. In Figure 3(d), both the coefficient of each index and the quality of image fusion are the highest, but the algorithm needs a large amount of computation and is too time-consuming. Figure 3(e) is the result of our nonlinear contourlet algorithm, in which index coefficients outperform wavelet transform and contourlet transform, and the fusion image quality is close to the NSCT; moreover, it greatly reduces the amount of calculation and has significant application value.Table 3
Indexes of performance evaluation.
Standard deviation
Correlation coefficients
Entropy
Mutual information
Running time
Wavelet fusion
50.48
0.64
6.78
4.16
1.608
Haar fusion
58.60
0.66
7.59
4.47
1.901
Contourlet fusion
59.32
0.72
7.63
4.52
3.021
Nonsubsampled contourlet fusion
65.01
0.73
7.72
4.59
5.561
Nonlinear contourlet fusion
63.16
0.72
7.70
4.55
1.254Figures4, 5, and 6 are another group of fusion results of CT and MRI medical images using different algorithms; indexes of the performance evaluation are shown in Tables 4, 5, and 6, respectively. Both from the subject visual perspective and from the object standard of the fused image estimate, the experiment result of our algorithm is better than wavelet and contourlet fusion algorithm. When compared with the nonsubsampled contourlet fusion algorithm, our algorithm has greatly enhanced the computing speed, without losing the fusion quality.Table 4
Index of performance evaluation.
Standard deviation
Correlation coefficients
Entropy
Mutual information
Running time
Wavelet fusion
80.14
0.75
2.24
2.15
2.013
Haar fusion
80.27
0.77
3.54
2.22
2.408
Contourlet fusion
81.30
0.80
4.39
2.27
4.211
Nonsubsampled contourlet fusion
81.48
0.84
5.23
3.32
9.054
Nonlinear contourlet fusion
81.39
0.81
5.14
2.91
1.701Table 5
Index of performance evaluation.
Standard deviation
Correlation coefficients
Entropy
Mutual information
Running time
Wavelet fusion
53.11
0.72
6.23
1.96
1.752
Haar fusion
57.84
0.75
6.30
2.03
1.805
Contourlet fusion
60.82
0.83
7.52
2.10
3.501
Nonsubsampled contourlet fusion
70.06
0.96
8.04
2.07
7.526
Nonlinear contourlet fusion
62.97
0.85
7.69
2.12
1.055Table 6
Index of performance evaluation.
Standard deviation
Correlation coefficients
Entropy
Mutual information
Running time
Wavelet fusion
44.97
0.73
5.78
4.80
2.805
Haar fusion
49.72
0.76
5.88
4.95
2.932
Contourlet fusion
51.26
0.80
6.35
5.42
4.921
Nonsubsampled contourlet fusion
59.06
0.88
6.60
5.67
10.594
Nonlinear contourlet fusion
52.33
0.82
6.42
5.44
1.824Figure 4
Fusion results’ comparison.
(a)
CT image
(b)
MRI image
(c)
Wavelet fusion
(d)
Haar fusion
(e)
Contourlet fusion
(f)
Nonsubsampled contourlet fusion
(g)
Nonlinear contourlet fusionFigure 5
Fusion results’ comparison.
(a)
CT image
(b)
PET image
(c)
Wavelet fusion
(d)
Haar fusion
(e)
Contourlet fusion
(f)
Nonsubsampled fusion
(g)
Nonlinear contourlet fusionFigure 6
Fusion results’ comparison.
(a)
CT image
(b)
MRI image
(c)
Wavelet fusion
(d)
Haar fusion
(e)
Contourlet fusion
(f)
Nonsubsampled fusion
(g)
Nonlinear contourlet fusion


## 4.1. Standard Deviation

The standard deviation (STD) reflects the contrast change of an image: the larger the value, the clearer the edge contour; the definition of STD is given as follows:(11)
S
T
D
=
∑
i
=
1
M
∑
j
=
1
N
f
i
,
j
-
f
^
2
M
×
N
.In the formula,f
i
,
j represents the result fusion image, while M
×
N represents the image f, and f
^ represents the mean value of f.


## 4.2. Correlation Coefficients

Correlation coefficient (CC) is a measure of the similarity of two images: the greater the correlation coefficient, the better the fusion effect; CC is defined as(12)
C
C
X
,
Y
=
∑
i
=
1
M
∑
j
=
1
N
X
i
,
j
-
X
-
Y
i
,
j
-
Y
-
∑
i
=
1
M
∑
j
=
1
N
X
i
,
j
-
X
-
2
∑
i
=
1
M
∑
j
=
1
N
Y
i
,
j
-
Y
-
2
.


## 4.3. Entropy

Entropy (EN) measures the amount of information maintained by the fused image. The larger the entropy value is, the more information the result image has. EN is defined as(13)
E
N
=
-
∑
0
255
P
i
x
log
2
⁡
p
i
x
,in whichP
i
x is defined as the normalized histogram of the variable x and ∑
0
255
P
i
x
=
1.


## 4.4. Mutual Information

Mutual information (MI) can be used to measure the mutual correlation or similarity of the two input images. The higher the mutual information of the fused image, the more information extracted from the original images and the better the fusion effect. For the two input discrete image signals, MI is defined as(14)
M
I
x
,
y
=
H
x
+
H
y
-
H
x
,
y
,in whichH
(
x
) is an entropy function; H
(
x
)
=
-
∑
0
255
P
i
x
log
2
⁡
p
i
(
x
), while H
x
,
y is a joint entropy function; H
(
x
,
y
)
=
-
∑
0
255
P
i
x
,
y
log
2
⁡
p
i
(
x
,
y
). P
i
(
x
,
y
) is defined as histogram of normalized distribution of x and  y and ∑
0
255
P
i
x
,
y
=
1.As shown in Table2, the maximum value of the standard deviation in the fused image can be obtained for less than 50% significant coefficients, which demonstrate sharper profile edges and the highest resolution at 50% nonlinear approximation. This may be the result of few significant coefficients which will in turn result in less extraction of edge contours; alternatively, too many significant coefficients will lead to the extraction of excess noise. Correlation coefficients, mutual information, and entropy have little effect on image fusion with various proportions of significant coefficients. The primary reason for this is that the low-frequency subband retains the profile information of the original image, while fused images do not achieve nonlinear approximation in the low-frequency region. Results indicate that fusion image quality demonstrates a nonlinear relationship to the proportion of significant coefficients. When the significant coefficient is 50%, the quality of image fusion is highest.Table 2
Indexes of performance evaluation.
Standard deviation
Correlation coefficients
Entropy
Mutual information
10% significant coefficients
50.48
0.66
7.68
4.51
30% significant coefficients
56.32
0.69
7.68
4.53
50% significant coefficients
63.16
0.70
7.70
4.54
70% significant coefficients
59.32
0.71
7.70
4.55
90% significant coefficients
57.16
0.73
7.72
4.55In order to verify the effectiveness of the algorithm, we compare our result with others from different researchers. Figure3 shows different fusion results for MRI and CT images with different algorithms, in which Figure 3(a) is the experimental result with fusion algorithm based on regional feature of wavelet coefficients [24], Figure 3(b) is experimental result based on Haar wavelet transform [25], Figure 3(c) is the result of coefficient weighted fusion algorithm after contourlet transform [19], Figure 3(d) is the fusion result based on the nonsubsampled contourlet transform (NSCT) and regional features [26], and Figure 3(e) is our experimental fusion result, which is an effect of regional image feature matching after 50% nonlinear estimation to the high-frequency coefficients by the contourlet transform. For the visual assessment, the fusion images (Figures 3(a) and 3(b)) are based on the wavelet transform methods; the color is shallow, and edge profile is not clear enough, which cannot reflect the source CT and MRI images of the details of the information. In Figure 3(c), with contourlet transform, the fusion image effect is slightly better than the wavelet method and can more clearly reflect the content information of the source images. In Figures 3(d) and 3(e), the fusion image not only inherits the bone tissue of the CT image but also can keep the soft tissue of the MRI image; moreover, the image edge detail can be more obvious after image fusion.Figure 3
Fusion results’ comparison.
(a)
CT image
(b)
MRI image
(c)
Wavelet fusion
(d)
Haar fusion
(e)
Contourlet fusion
(f)
Nonsubsampled contourlet fusion
(g)
Nonlinear contourlet fusionAnalyzing the results quantitatively with the indexes shown in Table3, the corresponding STD, CC, EN, and MI of Figures 3(a) and 3(b), which are based on the wavelet transform methods, are relatively small, which shows that the information has been lost in the process of image fusion and also shows a notable difference between the fusion image and the source images. It has been demonstrated that the ability of wavelet transform method to extract information from the source image is poor, and the fusion effect is not good. Figure 3(c) gets more directions in the high-frequency part by using contourlet transform method and can better deal with the direction of the details relative to the wavelet transform, so the coefficient of each index is superior to wavelet fusion. In Figure 3(d), both the coefficient of each index and the quality of image fusion are the highest, but the algorithm needs a large amount of computation and is too time-consuming. Figure 3(e) is the result of our nonlinear contourlet algorithm, in which index coefficients outperform wavelet transform and contourlet transform, and the fusion image quality is close to the NSCT; moreover, it greatly reduces the amount of calculation and has significant application value.Table 3
Indexes of performance evaluation.
Standard deviation
Correlation coefficients
Entropy
Mutual information
Running time
Wavelet fusion
50.48
0.64
6.78
4.16
1.608
Haar fusion
58.60
0.66
7.59
4.47
1.901
Contourlet fusion
59.32
0.72
7.63
4.52
3.021
Nonsubsampled contourlet fusion
65.01
0.73
7.72
4.59
5.561
Nonlinear contourlet fusion
63.16
0.72
7.70
4.55
1.254Figures4, 5, and 6 are another group of fusion results of CT and MRI medical images using different algorithms; indexes of the performance evaluation are shown in Tables 4, 5, and 6, respectively. Both from the subject visual perspective and from the object standard of the fused image estimate, the experiment result of our algorithm is better than wavelet and contourlet fusion algorithm. When compared with the nonsubsampled contourlet fusion algorithm, our algorithm has greatly enhanced the computing speed, without losing the fusion quality.Table 4
Index of performance evaluation.
Standard deviation
Correlation coefficients
Entropy
Mutual information
Running time
Wavelet fusion
80.14
0.75
2.24
2.15
2.013
Haar fusion
80.27
0.77
3.54
2.22
2.408
Contourlet fusion
81.30
0.80
4.39
2.27
4.211
Nonsubsampled contourlet fusion
81.48
0.84
5.23
3.32
9.054
Nonlinear contourlet fusion
81.39
0.81
5.14
2.91
1.701Table 5
Index of performance evaluation.
Standard deviation
Correlation coefficients
Entropy
Mutual information
Running time
Wavelet fusion
53.11
0.72
6.23
1.96
1.752
Haar fusion
57.84
0.75
6.30
2.03
1.805
Contourlet fusion
60.82
0.83
7.52
2.10
3.501
Nonsubsampled contourlet fusion
70.06
0.96
8.04
2.07
7.526
Nonlinear contourlet fusion
62.97
0.85
7.69
2.12
1.055Table 6
Index of performance evaluation.
Standard deviation
Correlation coefficients
Entropy
Mutual information
Running time
Wavelet fusion
44.97
0.73
5.78
4.80
2.805
Haar fusion
49.72
0.76
5.88
4.95
2.932
Contourlet fusion
51.26
0.80
6.35
5.42
4.921
Nonsubsampled contourlet fusion
59.06
0.88
6.60
5.67
10.594
Nonlinear contourlet fusion
52.33
0.82
6.42
5.44
1.824Figure 4
Fusion results’ comparison.
(a)
CT image
(b)
MRI image
(c)
Wavelet fusion
(d)
Haar fusion
(e)
Contourlet fusion
(f)
Nonsubsampled contourlet fusion
(g)
Nonlinear contourlet fusionFigure 5
Fusion results’ comparison.
(a)
CT image
(b)
PET image
(c)
Wavelet fusion
(d)
Haar fusion
(e)
Contourlet fusion
(f)
Nonsubsampled fusion
(g)
Nonlinear contourlet fusionFigure 6
Fusion results’ comparison.
(a)
CT image
(b)
MRI image
(c)
Wavelet fusion
(d)
Haar fusion
(e)
Contourlet fusion
(f)
Nonsubsampled fusion
(g)
Nonlinear contourlet fusion


## 5. Conclusions

This paper proposes an image fusion algorithm based on nonlinear approximation of contourlet transform and regional features applied to medical image processing, which combines nonlinear approximation of contourlet transform characteristics with the high-low frequency coefficient region feature algorithm. First, the algorithm retains the original low-frequency coefficients and then approximates high-frequency coefficients in order to extract the most nonlinear significant coefficients. Second, a coefficient matrix module is used in the low-frequency region to calculate the energy region near the center point and choose a reasonable fusion coefficient, which improves profile information retention of the original image. Finally, the rules of the salience/match measure with a threshold are used to calculate the center point of the region analysis feature and select the integration factor in the high-frequency region according to the most significant coefficients in order to better fuse image edge contours and texture. Via the experiment on CT and MRI images, the results show that the fusion image quality of our algorithm greatly outperforms the wavelet coefficients, Haar wavelet transform, and coefficient weighted contourlet transform and is very close to NSCT. Moreover, our algorithm has greatly improved the computation efficiency. Nonlinear contourlet transform has a certain filter and noise removal function, which outperforms the contourlet transform, while high-frequency component nonlinear feature extraction method retains features to maximum and has greatly improved the computing speed, which outperforms NSCT. In short, the method has certain advantages in both image quality and computation efficiency, so it can provide more reliable information to doctors.



---

*Source: 6807473-2017-01-24.xml*