# Lung Segmentation in 4D CT Volumes Based on Robust Active Shape Model Matching

Authors: Gurman Gill; Reinhard R. Beichel
Journal: International Journal of Biomedical Imaging
 (2015)

---

## Abstract

Dynamic and longitudinal lung CT imaging produce 4D lung image data sets, enabling applications like radiation treatment planning or assessment of response to treatment of lung diseases. In this paper, we present a 4D lung segmentation method that mutually utilizes all individual CT volumes to derive segmentations for each CT data set. Our approach is based on a 3D robust active shape model and extends it to fully utilize 4D lung image data sets. This yields an initial segmentation for the 4D volume, which is then refined by using a 4D optimal surface finding algorithm. The approach was evaluated on a diverse set of 152 CT scans of normal and diseased lungs, consisting of total lung capacity and functional residual capacity scan pairs. In addition, a comparison to a 3D segmentation method and a registration based 4D lung segmentation approach was performed. The proposed 4D method obtained an average Dice coefficient of0.9773
±
0.0254, which was statistically significantly better (p value ≪
0.001) than the 3D method (0.9659
±
0.0517). Compared to the registration based 4D method, our method obtained better or similar performance, but was 58.6% faster. Also, the method can be easily expanded to process 4D CT data sets consisting of several volumes.


---

## Body

## 1. Introduction

Applications like lung cancer radiotherapy planning [1], assessment of lung diseases like COPD [2], or dynamic lung ventilation studies [3] require the acquisition and subsequent analysis of 4D lung CT scans (e.g., two lung scans at different respiratory states). Most quantitative analysis approaches utilize image registration methods [4, 5] for 4D analysis. In order to achieve accurate results and reduce computation time, registration is typically only performed within a lung mask. Thus, for such approaches, the segmentation of each lung CT volume acquired is a prerequisite. This can be accomplished by utilizing standard 3D lung segmentation methods like the ones proposed in [6–10], which assume a large density difference between air-filled lung parenchyma and surrounding objects/tissues. However, since 4D imaging is mainly performed for the assessment and/or treatment of lung diseases, such simple methods frequently fail to perform well. Recently, 3D segmentation methods have been developed to deal with this issue, including approaches that utilize an atlas-based segmentation-by-registration scheme [11], an error-correcting hybrid system [12], a shape “break-and-repair” strategy [13], and a 3D robust active shape model (RASM) [14, 15]. However, none of these approaches takes advantage of 4D lung CT scans and thus requires lungs to be segmented individually. This can be problematic, especially when segmenting pairs of total lung capacity (TLC) and functional residual capacity (FRC) lung scans, because (diseased) lungs at FRC are typically more difficult to segment than lungs imaged at TLC. Consequently, algorithms that simultaneously segment lungs in all available CT volumes are more promising.Work on 4D lung segmentation techniques is scant. Wilms et al. [16] adopted a 4D statistical shape model that was originally developed by Perperidis et al. [17] for segmentation of gated cardiac image sequences. A limitation of this approach is that it is based on standard least squares active shape model (ASM) matching, which is known to be affected by outliers [14, 18]. Consequently, disease induced changes of lung tissue (e.g., density) or artifacts resulting from sorting algorithm errors in case of free-breathing CT lung imaging can adversely impact model matching.In our previous work, Sun et al. [19] introduced a 4D lung segmentation method based on 4D optimal surface finding (OSF). The approach requires a rough initial lung segmentation, which was obtained by applying a 3D RASM [14] to a TLC lung scan and transferring this segmentation by means of a nonrigid image registration to the corresponding FRC scan (Figure 1(b)). This approach has some potential shortcomings. First, the initial, rough segmentation step does not take full advantage of the available 4D CT data. Consequently, if the initial lung segmentation is inaccurate, the error is propagated to the other volume by the algorithm. Second, for many applications (e.g., segmentation of longitudinal TLC volumes), it is not obvious which volume should be utilized for 3D RASM segmentation to achieve good segmentation performance. Third, the registration step is quite time-consuming.Figure 1
RASM-based approaches for segmenting lungs in a 4D CT volume (represented by 3D volumes A and B). (a) 3D segmentation method [14]. (b) Registration based 4D segmentation method [19]. (c) Proposed 4D segmentation approach with new 4D RASM (Section 3.1).
(a)
(b)
(c)In this paper, we address these limitations by proposing a new 4D RASM model matching step that replaces the combination of single 3D RASM segmentation and subsequent registration to other volumes as proposed by Sun et al. [19]. In addition, we provide an extensive study, comparing the 3D base method published by Sun et al. [14] applied to each CT scan independently (Figure 1(a)), the registration based 4D version [19] (Figure 1(b)), and the proposed approach (Figure 1(c)) on a diverse set of 4D CT data, consisting of TLC and FRC scan pairs of normal and diseased lungs.


## 2. Prior Work

The proposed method extends our 3D RASM method [14] to mutual segmentation of lungs in 4D CT data. Thus, we briefly outline the RASM fitting process for a single 3D volume.The RASM consists of a point distribution model (PDM) that captures the variation in lung shapes and a robust matching approach that iteratively fits the model to a lung CT scan to perform a segmentation. The PDM is constructed separately for left and right lungs fromN lung volume training data sets that have m corresponding points (landmarks) [14]. An instance of a left or right lung shape is generated from the corresponding PDM by the linear model(1)
x
=
x
¯
+
P
b
,where x
¯ is the mean lung shape vector, P denotes the shape eigenvector matrix, and b represents the shape coefficients.The matching process begins with automatically placing the mean lung shapex
¯ in the target CT volume based on a ribs detection step [14]. The model shape points are then updated to y based on a gradient based cost function. For this purpose, a robust matching step is utilized to prevent that outliers are used during the model matching process. It is based on a robust PCA coefficient estimation method, which utilizes subsets of landmark points ω
i
,
j and a voting scheme [14]. During matching, for each of these subsets a reconstruction error e
ω
i
,
j is calculated, which is then being used to determine the inliers update points y
~ of y (note the notation for specifying inliers using ~; e.g., y
~ corresponds to the set y
{
p
i
=
1
} in [14] and is used here instead for the sake of clarity). Subsequently, y
~ is utilized to update the shape coefficients by calculating(2)
b
=
P
~
T
T
y
~
-
x
¯
~
,where T is the pose transformation matrix for mapping points from target image coordinate frame to model coordinate frame and x
¯
~ denotes the points corresponding to inliers in the mean lung shape x
¯. P
~ refers to the columns corresponding to inliers in the shape eigenvector matrix P. A new instance of the model is calculated using (1), which is transformed to the image space by T
-
1. The model shape points are then iteratively updated until convergence.Once the robust matching process is finished, the resulting RASM segmentation is used as an initial shape for a graph-based optimal surface finding (OSF) algorithm to further refine the segmentation [14] (Figure 1(a)).


## 3. Methods

Our method for generating a 4D lung segmentation (Figure1(c)) is based on fitting a 3D RASM mutually to 4D volume data (Section 3.1), followed by a 4D OSF segmentation step (Section 3.2). The results of all the different processing stages are depicted in Figure 2. Below, we describe the segmentation process in detail for a TLC and FRC lung scan pair, but the approach would also work for other respiratory states or longitudinal scans and can be expanded to more than two lung volumes.Figure 2
Illustration of intermediate and final results of the proposed 4D lung segmentation approach. (a) 4D CT data consisting of a TLC and FRC lung scan pair to be segmented. (b) Model initialization. (c-d) 4D RASM segmentation with (c) mutual segmentations and (d) result after the constrained adaptation step. (e) 4D OSF segmentation result.
(a)
(b)
(c)
(d)
(e)In addition to the main processing steps described in Sections3.1 and 3.2, the following two preprocessing steps are performed. First, a modified system of the airway tree segmentation method [20] is utilized to extract the trachea and main bronchi, which are then dilated using a radius of 2 voxels. These locations are assigned a value of 50 HU in order to make them unattractive for RASM and OSF segmentation. Second, an overlap between left and right lung segmentations is avoided by detecting the thin tissue layer between the lungs, as described by Gill et al. [21].


### 3.1.
4D RASM Segmentation

For model-based segmentation, a lung PDM is constructed from 75 TLC and 75 FRC normal lung CT scan pairs, which are not part of the image data utilized for method evaluation (Section4.1). Note that model building is done separately for right and left lungs. Utilizing the right or left PDM, 4D RASM segmentation consists of the following main processing steps (Figure 3).Figure 3
Flowchart showing the steps involved in fitting the model to 4D CT data. The independent and mutual steps are identified in the iterative model fitting process.(a) Model Initialization. The mean lung model x
¯ is placed independently in the target TLC and FRC volumes (Figure 2(b)). For this purpose, a ribs detection method [14] is applied on the respective volumes.(b) Iterative Model Fitting. The matching steps (i) to (v) given below are repeated for 90 iterations, which are sufficient to achieve model convergence (Figure 2(c)). Alternatively, a convergence criterion could be used.(i)
Updating Shape Points. Utilizing a gray-value gradient based cost function [14] of TLC and FRC volumes, the model shape points are independently updated in the TLC and FRC volumes, resulting in y
t
l
c and y
f
r
c, respectively.
(ii)
Robustly Estimating Mutual Inlier Update Points. Update point sets y
t
l
c and y
f
r
c are used to calculate e
ω
i
,
j
t
l
c and e
ω
i
,
j
f
r
c, respectively, which is similar to that described in [14]. However, after this step, a mutual reconstruction error is calculated with(3)
e
ω
i
,
j
mutual
=
e
ω
i
,
j
tlc
+
e
ω
i
,
j
frc
2to enable mutual inlier estimation. Thus, e
ω
i
,
j
m
u
t
u
a
l is used in the voting scheme described in [14] instead of individual reconstruction errors e
ω
i
,
j
t
l
c or e
ω
i
,
j
f
r
c. The outcomes of the voting process are inlier update point sets y
~
t
l
c and y
~
f
r
c. Note that, while y
~
t
l
c and y
~
f
r
c are different, they have the same cardinality and correspond to the same landmark points of the lung PDM.
(iii)
Computing Mutual Shape Coefficients. The inlier point sets y
~
t
l
c and y
~
f
r
c are independently transformed to the model coordinate frame by using pose transformation matrices T
t
l
c and T
f
r
c, respectively. Each transformation is derived from a Procrustes analysis between inlier sets (y
~
t
l
c and y
~
f
r
c) and corresponding mean model (x
¯
~) in model coordinate space. The shape coefficients b
m
u
t
u
a
l are computed using the average of the transformed inliers(4)
b
mutual
=
P
~
T
T
tlc
y
~
tlc
+
T
frc
y
~
frc
2
-
x
¯
~
.
(iv)
Generating a New Model Instance. A new instance of the model, which is used to represent the lung in TLC and FRC scans, is calculated using (1) and b
m
u
t
u
a
l.
(v)
Transforming the Model. The model is transformed back to TLC and FRC volumes using T
t
l
c
-
1 and T
f
r
c
-
1, respectively.(c) Constrained Model Adaptation. After the 4D fitting process converges, a single lung shape with individual transformation matrices T
t
l
c and T
f
r
c results, which matches the lungs in TLC and FRC scans. However, the transformations only account for isotropic scaling. Thus, the fitted models will not be perfectly aligned with the image data, because the difference in TLC and FRC lung shapes cannot be explained by an isotropic scale factor. To obtain a better alignment, we subsequently allow the shape coefficients to individually adapt to the target images by continuing the RASM fitting process independently in both volumes for ten iterations (Figure 2(d)). Note that this adaptation is done in a constrained manner, only allowing a subset of model coefficients to change within certain limits to avoid major divergence of TLC and FRC models. The subset of model coefficients (sorted in decreasing order of their eigenvalues) is defined by the coefficients whose eigenvalues account for 80% of shape variation. In our case, this resulted in a set of 22 coefficients out of 150, which were allowed to change by a maximum of 0.5 times the standard deviation σ in the respective eigenvalues. Thus, the final shape coefficients b
a
d
a
p
t
e
d after the individual adaptation step for the TLC or FRC volume are limited to (5)
b
mutual
l
±
0.5
σ
l
if
l
≤
22
b
mutual
l
o
t
h
e
r
w
i
s
e
.The parameters constraining the model were selected conservatively, and we found that small parameter variations have little impact on the overall lung segmentation.


### 3.2.
4D OSF Segmentation

After the initial model-based segmentations are created for TLC and FRC volumes, they are refined using the 4D OSF method [19], resulting in the final 4D lung segmentation (Figure 2(e)). For this purpose, the same parameter settings as proposed by Sun et al. [19] were utilized.


## 3.1.
4D RASM Segmentation

For model-based segmentation, a lung PDM is constructed from 75 TLC and 75 FRC normal lung CT scan pairs, which are not part of the image data utilized for method evaluation (Section4.1). Note that model building is done separately for right and left lungs. Utilizing the right or left PDM, 4D RASM segmentation consists of the following main processing steps (Figure 3).Figure 3
Flowchart showing the steps involved in fitting the model to 4D CT data. The independent and mutual steps are identified in the iterative model fitting process.(a) Model Initialization. The mean lung model x
¯ is placed independently in the target TLC and FRC volumes (Figure 2(b)). For this purpose, a ribs detection method [14] is applied on the respective volumes.(b) Iterative Model Fitting. The matching steps (i) to (v) given below are repeated for 90 iterations, which are sufficient to achieve model convergence (Figure 2(c)). Alternatively, a convergence criterion could be used.(i)
Updating Shape Points. Utilizing a gray-value gradient based cost function [14] of TLC and FRC volumes, the model shape points are independently updated in the TLC and FRC volumes, resulting in y
t
l
c and y
f
r
c, respectively.
(ii)
Robustly Estimating Mutual Inlier Update Points. Update point sets y
t
l
c and y
f
r
c are used to calculate e
ω
i
,
j
t
l
c and e
ω
i
,
j
f
r
c, respectively, which is similar to that described in [14]. However, after this step, a mutual reconstruction error is calculated with(3)
e
ω
i
,
j
mutual
=
e
ω
i
,
j
tlc
+
e
ω
i
,
j
frc
2to enable mutual inlier estimation. Thus, e
ω
i
,
j
m
u
t
u
a
l is used in the voting scheme described in [14] instead of individual reconstruction errors e
ω
i
,
j
t
l
c or e
ω
i
,
j
f
r
c. The outcomes of the voting process are inlier update point sets y
~
t
l
c and y
~
f
r
c. Note that, while y
~
t
l
c and y
~
f
r
c are different, they have the same cardinality and correspond to the same landmark points of the lung PDM.
(iii)
Computing Mutual Shape Coefficients. The inlier point sets y
~
t
l
c and y
~
f
r
c are independently transformed to the model coordinate frame by using pose transformation matrices T
t
l
c and T
f
r
c, respectively. Each transformation is derived from a Procrustes analysis between inlier sets (y
~
t
l
c and y
~
f
r
c) and corresponding mean model (x
¯
~) in model coordinate space. The shape coefficients b
m
u
t
u
a
l are computed using the average of the transformed inliers(4)
b
mutual
=
P
~
T
T
tlc
y
~
tlc
+
T
frc
y
~
frc
2
-
x
¯
~
.
(iv)
Generating a New Model Instance. A new instance of the model, which is used to represent the lung in TLC and FRC scans, is calculated using (1) and b
m
u
t
u
a
l.
(v)
Transforming the Model. The model is transformed back to TLC and FRC volumes using T
t
l
c
-
1 and T
f
r
c
-
1, respectively.(c) Constrained Model Adaptation. After the 4D fitting process converges, a single lung shape with individual transformation matrices T
t
l
c and T
f
r
c results, which matches the lungs in TLC and FRC scans. However, the transformations only account for isotropic scaling. Thus, the fitted models will not be perfectly aligned with the image data, because the difference in TLC and FRC lung shapes cannot be explained by an isotropic scale factor. To obtain a better alignment, we subsequently allow the shape coefficients to individually adapt to the target images by continuing the RASM fitting process independently in both volumes for ten iterations (Figure 2(d)). Note that this adaptation is done in a constrained manner, only allowing a subset of model coefficients to change within certain limits to avoid major divergence of TLC and FRC models. The subset of model coefficients (sorted in decreasing order of their eigenvalues) is defined by the coefficients whose eigenvalues account for 80% of shape variation. In our case, this resulted in a set of 22 coefficients out of 150, which were allowed to change by a maximum of 0.5 times the standard deviation σ in the respective eigenvalues. Thus, the final shape coefficients b
a
d
a
p
t
e
d after the individual adaptation step for the TLC or FRC volume are limited to (5)
b
mutual
l
±
0.5
σ
l
if
l
≤
22
b
mutual
l
o
t
h
e
r
w
i
s
e
.The parameters constraining the model were selected conservatively, and we found that small parameter variations have little impact on the overall lung segmentation.


## 3.2.
4D OSF Segmentation

After the initial model-based segmentations are created for TLC and FRC volumes, they are refined using the 4D OSF method [19], resulting in the final 4D lung segmentation (Figure 2(e)). For this purpose, the same parameter settings as proposed by Sun et al. [19] were utilized.


## 4. Evaluation

### 4.1. Image Data

For evaluation,152 multidetector computed tomography (MDCT) thorax scans of lungs from 4 different sets S
n
o
r
m
a
l, S
a
s
t
h
m
a, S
C
O
P
D, and S
m
i
x with no significant abnormalities (normal), asthma (both severe and nonsevere), chronic obstructive pulmonary disease (COPD, GOLD 1 to 4), and a mixture of different lung diseases, respectively, were utilized. The total number of scans in sets S
n
o
r
m
a
l, S
a
s
t
h
m
a, S
C
O
P
D, and S
m
i
x were 40, 36, 36, and 40, respectively. All the four sets contained pairs of TLC (volume A) and FRC (volume B) images. The image sizes varied from 512
×
512
×
351 to 512
×
512
×
781 voxels with a mean size of 512
×
512
×
580 voxels. The slice thickness of images ranged from 0.5 to 0.63 mm (mean: 0.52 mm) and the in-plane resolution from 0.49
×
0.49 to 0.91
×
0.91 mm (mean: 0.64
×
0.64 mm).


### 4.2. Experimental Setup

For all test data sets, an independent reference standard was generated. Manual segmentation of a whole lung is time-consuming, and due to the large number of 152 test CT scans, we utilize a sampling approach, which is similar to that utilized in [6, 11, 12], to reduce the substantial effort required for manual inspection and segmentation. Thus, for every tenth axial slice, a trained expert generated a reference segmentation under the supervision of a pulmonologist, resulting in a dense sampling of the lung volume with between 41 and 64 labeled slices for each data set (Figure 4). The same sampling approach was applied to the segmentation result to be evaluated. Based on the sampled volumes, the Dice coefficient D [22] was calculated. In addition, the mean unsigned distance error d [22] was computed with respect to the reference in all axial slices where a reference standard and segmentation result were both available. Subsequently, the average of all these locations was calculated per data set and reported.Figure 4
Example showing the location and density of axial reference segmentations in relation to lung anatomy.In the following sections, the proposed method (Figure1(c)) will be denoted by M
4
D. In addition, two other methods will be utilized for comparison. M
3
D will be utilized to denote the 3D approach proposed by Sun et al. [14] (Figure 1(a)). The 4D method of Sun et al. [19] (Figure 1(b)) was used in two variants; the variant where volume A (TLC) is registered to volume B (FRC) will be denoted by M
4
D
r
e
g
A
B, and the variant where volume B (FRC) is registered to volume A (TLC) will be denoted by M
4
D
r
e
g
B
A. Investigating these two variants allows us to assess and compare performance in situations with different but unknown respiratory state (e.g., longitudinal lung image data). For all methods utilized, the standard parameter setting as described in respective papers was used. Unless otherwise mentioned, all reported results refer to the final (OSF) segmentation.A paired permutation test [23] was utilized for determining statistical significance, because it does not make assumptions about the distribution of the underlying data and a paired t-test or paired signed rank test was not applicable to our data.


## 4.1. Image Data

For evaluation,152 multidetector computed tomography (MDCT) thorax scans of lungs from 4 different sets S
n
o
r
m
a
l, S
a
s
t
h
m
a, S
C
O
P
D, and S
m
i
x with no significant abnormalities (normal), asthma (both severe and nonsevere), chronic obstructive pulmonary disease (COPD, GOLD 1 to 4), and a mixture of different lung diseases, respectively, were utilized. The total number of scans in sets S
n
o
r
m
a
l, S
a
s
t
h
m
a, S
C
O
P
D, and S
m
i
x were 40, 36, 36, and 40, respectively. All the four sets contained pairs of TLC (volume A) and FRC (volume B) images. The image sizes varied from 512
×
512
×
351 to 512
×
512
×
781 voxels with a mean size of 512
×
512
×
580 voxels. The slice thickness of images ranged from 0.5 to 0.63 mm (mean: 0.52 mm) and the in-plane resolution from 0.49
×
0.49 to 0.91
×
0.91 mm (mean: 0.64
×
0.64 mm).


## 4.2. Experimental Setup

For all test data sets, an independent reference standard was generated. Manual segmentation of a whole lung is time-consuming, and due to the large number of 152 test CT scans, we utilize a sampling approach, which is similar to that utilized in [6, 11, 12], to reduce the substantial effort required for manual inspection and segmentation. Thus, for every tenth axial slice, a trained expert generated a reference segmentation under the supervision of a pulmonologist, resulting in a dense sampling of the lung volume with between 41 and 64 labeled slices for each data set (Figure 4). The same sampling approach was applied to the segmentation result to be evaluated. Based on the sampled volumes, the Dice coefficient D [22] was calculated. In addition, the mean unsigned distance error d [22] was computed with respect to the reference in all axial slices where a reference standard and segmentation result were both available. Subsequently, the average of all these locations was calculated per data set and reported.Figure 4
Example showing the location and density of axial reference segmentations in relation to lung anatomy.In the following sections, the proposed method (Figure1(c)) will be denoted by M
4
D. In addition, two other methods will be utilized for comparison. M
3
D will be utilized to denote the 3D approach proposed by Sun et al. [14] (Figure 1(a)). The 4D method of Sun et al. [19] (Figure 1(b)) was used in two variants; the variant where volume A (TLC) is registered to volume B (FRC) will be denoted by M
4
D
r
e
g
A
B, and the variant where volume B (FRC) is registered to volume A (TLC) will be denoted by M
4
D
r
e
g
B
A. Investigating these two variants allows us to assess and compare performance in situations with different but unknown respiratory state (e.g., longitudinal lung image data). For all methods utilized, the standard parameter setting as described in respective papers was used. Unless otherwise mentioned, all reported results refer to the final (OSF) segmentation.A paired permutation test [23] was utilized for determining statistical significance, because it does not make assumptions about the distribution of the underlying data and a paired t-test or paired signed rank test was not applicable to our data.


## 5. Results

### 5.1. Segmentation Performance

Our novel 4D RASM matching approach (without final OSF segmentation step) showed an average Dice coefficient of0.9468
±
0.0318. In contrast, the standard 3D RASM approach resulted in an average Dice coefficient of 0.9391
±
0.0525. The 4D RASM showed a statistically significant improvement (p value ≪
0.001) compared against 3D RASM.Tables1 and 2 summarize the resulting final (OSF) segmentation performance with corresponding p values, comparing results of M
3
D and M
4
D. In both tables, M
4
D shows statistically significant improvement in each data set and for TLC and FRC scans. Figure 5 depicts some examples of segmentations (one from each test data set) obtained by methods M
3
D and M
4
D.Table 1
Dice coefficientD for methods M
3
D and M
4
D. Statistically significant p values are marked with ∗.
Set
M
3
D
M
4
D
p value
S
normal
0.9822 ± 0.0129
0.9850 ± 0.0058
≪0.001
∗
S
asthma
0.9667 ± 0.0437
0.9789 ± 0.0108
≪0.001
∗
S
COPD
0.9685 ± 0.0514
0.9839 ± 0.0068
≪0.001
∗
S
mix
0.9467 ± 0.0732
0.9623 ± 0.0444
0.012
∗
TLC
0.9808 ± 0.0142
0.9846 ± 0.0103
≪0.001
∗
FRC
0.9510 ± 0.0687
0.9700 ± 0.0329
≪0.001
∗
ALL
0.9659 ± 0.0517
0.9773 ± 0.0254
≪0.001
∗Table 2
Mean unsigned distance errord in millimeters for methods M
3
D and M
4
D. Statistically significant p values are marked with ∗.
Set
M
3
D
M
4
D
p value
S
normal
1.20 ± 0.77
0.89 ± 0.44
≪0.001
∗
S
asthma
1.68 ± 1.25
1.14 ± 0.50
≪0.001
∗
S
COPD
1.66 ± 1.90
1.00 ± 0.56
≪0.001
∗
S
mix
2.77 ± 4.42
1.81 ± 1.45
≪0.001
∗
TLC
1.16 ± 0.68
0.88 ± 0.53
≪0.001
∗
FRC
2.52 ± 3.50
1.56 ± 1.11
≪0.001
∗
ALL
1.84 ± 2.61
1.22 ± 0.93
≪0.001
∗Figure 5
Comparison of segmentation results generated with methodsM
3
D and M
4
D. Coronal CT cross sections with marked lung boundaries in combination with corresponding 3D lung mesh models are shown to enable locating and assessing segmentation errors as well as differences between results.Tables3 and 4 compare the final lung segmentation Dice coefficient of M
4
D to M
4
D
r
e
g
A
B and M
4
D
r
e
g
B
A, respectively. Overall, M
4
D and M
4
D
r
e
g
A
B were found to be equivalent (no statistically significant difference), but M
4
D was found to be significantly better compared to M
4
D
r
e
g
B
A. Figure 6 provides a comparison of final Dice coefficients in form of box plots for all methods and separated by respiratory state. A comparison of results generated with M
4
D, M
4
D
r
e
g
A
B, and M
4
D
r
e
g
B
A is shown in Figure 7.Table 3
Dice coefficientD for methods M
4
DregAB and M
4
D. Statistically significant p values are marked with ∗.
Set
M
4
DRegAB
M
4
D
p value
TLC
0.9834 ± 0.0127
0.9846 ± 0.0103
8.92
e
-
03
∗
FRC
0.9718 ± 0.0233
0.9700 ± 0.0329
2.07
e
-
01
ALL
0.9776 ± 0.0196
0.9773 ± 0.0254
7.97
e
-
01Table 4
Dice coefficientD for methods M
4
DregBA and M
4
D. Statistically significant p values are marked with ∗.
Set
M
4
DregBA
M
4
D
p value
TLC
0.9687 ± 0.0579
0.9846 ± 0.0103
≪0.001
∗
FRC
0.9539 ± 0.0711
0.9700 ± 0.0329
≪0.001
∗
ALL
0.9613 ± 0.0651
0.9773 ± 0.0254
≪0.001
∗Figure 6
Box plots of Dice coefficients for methodsM
3
D, M
4
D
r
e
g
A
B, M
4
D
r
e
g
B
A, and M
4
D on (a) TLC and (b) FRC lung scans.
(a)
(b)Figure 7
Examples of segmentation results generated with methodsM
4
D, M
4
D
r
e
g
A
B, and M
4
D
r
e
g
B
A.


### 5.2. Computing Time

Segmentation withM
3
D took 13.21 minutes on average for TLC and FRC data sets combined (TLC: 6.73 minutes, FRC: 6.48 minutes). Method M
4
D required 12.22 minutes per 4D case, on average. Compared to M
3
D, the reduction in computing time was primarily achieved due to synergies of 4D processing. Approaches M
4
D
r
e
g
A
B and M
4
D
r
e
g
A
B took 29.49 minutes per 4D case, on average, where the registration procedure contributed to about 20 minutes of computing time.


## 5.1. Segmentation Performance

Our novel 4D RASM matching approach (without final OSF segmentation step) showed an average Dice coefficient of0.9468
±
0.0318. In contrast, the standard 3D RASM approach resulted in an average Dice coefficient of 0.9391
±
0.0525. The 4D RASM showed a statistically significant improvement (p value ≪
0.001) compared against 3D RASM.Tables1 and 2 summarize the resulting final (OSF) segmentation performance with corresponding p values, comparing results of M
3
D and M
4
D. In both tables, M
4
D shows statistically significant improvement in each data set and for TLC and FRC scans. Figure 5 depicts some examples of segmentations (one from each test data set) obtained by methods M
3
D and M
4
D.Table 1
Dice coefficientD for methods M
3
D and M
4
D. Statistically significant p values are marked with ∗.
Set
M
3
D
M
4
D
p value
S
normal
0.9822 ± 0.0129
0.9850 ± 0.0058
≪0.001
∗
S
asthma
0.9667 ± 0.0437
0.9789 ± 0.0108
≪0.001
∗
S
COPD
0.9685 ± 0.0514
0.9839 ± 0.0068
≪0.001
∗
S
mix
0.9467 ± 0.0732
0.9623 ± 0.0444
0.012
∗
TLC
0.9808 ± 0.0142
0.9846 ± 0.0103
≪0.001
∗
FRC
0.9510 ± 0.0687
0.9700 ± 0.0329
≪0.001
∗
ALL
0.9659 ± 0.0517
0.9773 ± 0.0254
≪0.001
∗Table 2
Mean unsigned distance errord in millimeters for methods M
3
D and M
4
D. Statistically significant p values are marked with ∗.
Set
M
3
D
M
4
D
p value
S
normal
1.20 ± 0.77
0.89 ± 0.44
≪0.001
∗
S
asthma
1.68 ± 1.25
1.14 ± 0.50
≪0.001
∗
S
COPD
1.66 ± 1.90
1.00 ± 0.56
≪0.001
∗
S
mix
2.77 ± 4.42
1.81 ± 1.45
≪0.001
∗
TLC
1.16 ± 0.68
0.88 ± 0.53
≪0.001
∗
FRC
2.52 ± 3.50
1.56 ± 1.11
≪0.001
∗
ALL
1.84 ± 2.61
1.22 ± 0.93
≪0.001
∗Figure 5
Comparison of segmentation results generated with methodsM
3
D and M
4
D. Coronal CT cross sections with marked lung boundaries in combination with corresponding 3D lung mesh models are shown to enable locating and assessing segmentation errors as well as differences between results.Tables3 and 4 compare the final lung segmentation Dice coefficient of M
4
D to M
4
D
r
e
g
A
B and M
4
D
r
e
g
B
A, respectively. Overall, M
4
D and M
4
D
r
e
g
A
B were found to be equivalent (no statistically significant difference), but M
4
D was found to be significantly better compared to M
4
D
r
e
g
B
A. Figure 6 provides a comparison of final Dice coefficients in form of box plots for all methods and separated by respiratory state. A comparison of results generated with M
4
D, M
4
D
r
e
g
A
B, and M
4
D
r
e
g
B
A is shown in Figure 7.Table 3
Dice coefficientD for methods M
4
DregAB and M
4
D. Statistically significant p values are marked with ∗.
Set
M
4
DRegAB
M
4
D
p value
TLC
0.9834 ± 0.0127
0.9846 ± 0.0103
8.92
e
-
03
∗
FRC
0.9718 ± 0.0233
0.9700 ± 0.0329
2.07
e
-
01
ALL
0.9776 ± 0.0196
0.9773 ± 0.0254
7.97
e
-
01Table 4
Dice coefficientD for methods M
4
DregBA and M
4
D. Statistically significant p values are marked with ∗.
Set
M
4
DregBA
M
4
D
p value
TLC
0.9687 ± 0.0579
0.9846 ± 0.0103
≪0.001
∗
FRC
0.9539 ± 0.0711
0.9700 ± 0.0329
≪0.001
∗
ALL
0.9613 ± 0.0651
0.9773 ± 0.0254
≪0.001
∗Figure 6
Box plots of Dice coefficients for methodsM
3
D, M
4
D
r
e
g
A
B, M
4
D
r
e
g
B
A, and M
4
D on (a) TLC and (b) FRC lung scans.
(a)
(b)Figure 7
Examples of segmentation results generated with methodsM
4
D, M
4
D
r
e
g
A
B, and M
4
D
r
e
g
B
A.


## 5.2. Computing Time

Segmentation withM
3
D took 13.21 minutes on average for TLC and FRC data sets combined (TLC: 6.73 minutes, FRC: 6.48 minutes). Method M
4
D required 12.22 minutes per 4D case, on average. Compared to M
3
D, the reduction in computing time was primarily achieved due to synergies of 4D processing. Approaches M
4
D
r
e
g
A
B and M
4
D
r
e
g
A
B took 29.49 minutes per 4D case, on average, where the registration procedure contributed to about 20 minutes of computing time.


## 6. Discussion

The main advantage of our 4D approachM
4
D is that it utilizes both lung volumes acquired at different respirator states for segmentation during all main processing stages, which is in contrast to the standard M
3
D method and 4D variants M
4
D
r
e
g
A
B and M
4
D
r
e
g
B
A. The results presented in Section 5 clearly demonstrate this advantage.


### 6.1. Comparison ofM
4
D with M
3
D

When compared to the 3D variant, statistically significant lung segmentation performance improvements, independent of test set, respiratory state, and performance metric, were observed (Tables1 and 2). This is also clearly demonstrated by the examples shown in Figure 5. As shown in Tables 1 and 2, the observed gain in segmentation accuracy with M
4
D was larger for cases with lung disease (test sets S
a
s
t
h
m
a, S
C
O
P
D, and S
m
i
x) compared to normal cases (test set S
n
o
r
m
a
l). The better segmentation performance of M
4
D is expected, because it addresses several weaknesses of M
3
D like problems with model initialization, which can cause the model to converge locally to other structures than lung boundaries. As Figure 6 as well as Tables 1 and 2 show, gains achieved with M
4
D are higher for lungs imaged at FRC, which are generally more difficult to segment. Also, 4D processing reduces the computing time by 7.5% compared to sequential 3D processing.


### 6.2. Comparison ofM
4
D with M
4
D
r
e
g
A
B and M
4
D
r
e
g
B
A

Overall, segmentation performance ofM
4
D and M
4
D
r
e
g
A
B was found to be equivalent (Table 3), while the comparison between M
4
D and M
4
D
r
e
g
B
A (Table 4) showed a statistically significant improvement for our proposed M
4
D approach. Also, compared to M
4
D
r
e
g
A
B and M
4
D
r
e
g
B
A, our M
4
D method showed a reduction in computing time by 58.6%. Thus, it is preferable to M
4
D
r
e
g
A
B and M
4
D
r
e
g
B
A, especially when the exact respiratory states are unknown and picking a lung scan with lower lung volume (exhale) as starting point (Volume A in Figure 1(b)) could potentially adversely impact segmentation performance. Figure 7 depicts examples where either M
4
D
r
e
g
A
B or M
4
D
r
e
g
B
A produces a local segmentation error, but M
4
D avoids such problems.


### 6.3. Possible Improvements and Extensions

As can be seen in Figure6, the proposed approach reduces the number and/or severity of outlier cases. However, some room for improvement still exists, which we will address in future work. For example, a better FRC model initialization would help in further improving overall segmentation performance of 3D and 4D methods, but we expect that M
4
D would still perform better, because it utilizes both scans for model matching, which offers increased robustness.Our method can be expanded to handle processing of more than two lungs scans at the same time. This can be done by extending (3) and (4) accordingly. Another advantage is that M
4
D does not require any prior knowledge of the breathing state of the lungs in individual CT scans, because it does not make any assumptions about respiratory state (e.g., breathing sequence). An example for processing longitudinal lung CT scans in the context of cancer treatment planning/assessment is provided in Figure 8. Note that, due to lung disease (cancer), patient compliance with the utilized imaging protocol (e.g., acquisition at TLC) cannot be assumed.Figure 8
Comparison of results of methodsM
4
D, M
3
D, M
4
D
r
e
g
A
B, and M
4
D
r
e
g
B
A on a longitudinal CT volume pair. The time between scans A and B was 19 months, and the imaging protocol was quite different.


## 6.1. Comparison ofM
4
D with M
3
D

When compared to the 3D variant, statistically significant lung segmentation performance improvements, independent of test set, respiratory state, and performance metric, were observed (Tables1 and 2). This is also clearly demonstrated by the examples shown in Figure 5. As shown in Tables 1 and 2, the observed gain in segmentation accuracy with M
4
D was larger for cases with lung disease (test sets S
a
s
t
h
m
a, S
C
O
P
D, and S
m
i
x) compared to normal cases (test set S
n
o
r
m
a
l). The better segmentation performance of M
4
D is expected, because it addresses several weaknesses of M
3
D like problems with model initialization, which can cause the model to converge locally to other structures than lung boundaries. As Figure 6 as well as Tables 1 and 2 show, gains achieved with M
4
D are higher for lungs imaged at FRC, which are generally more difficult to segment. Also, 4D processing reduces the computing time by 7.5% compared to sequential 3D processing.


## 6.2. Comparison ofM
4
D with M
4
D
r
e
g
A
B and M
4
D
r
e
g
B
A

Overall, segmentation performance ofM
4
D and M
4
D
r
e
g
A
B was found to be equivalent (Table 3), while the comparison between M
4
D and M
4
D
r
e
g
B
A (Table 4) showed a statistically significant improvement for our proposed M
4
D approach. Also, compared to M
4
D
r
e
g
A
B and M
4
D
r
e
g
B
A, our M
4
D method showed a reduction in computing time by 58.6%. Thus, it is preferable to M
4
D
r
e
g
A
B and M
4
D
r
e
g
B
A, especially when the exact respiratory states are unknown and picking a lung scan with lower lung volume (exhale) as starting point (Volume A in Figure 1(b)) could potentially adversely impact segmentation performance. Figure 7 depicts examples where either M
4
D
r
e
g
A
B or M
4
D
r
e
g
B
A produces a local segmentation error, but M
4
D avoids such problems.


## 6.3. Possible Improvements and Extensions

As can be seen in Figure6, the proposed approach reduces the number and/or severity of outlier cases. However, some room for improvement still exists, which we will address in future work. For example, a better FRC model initialization would help in further improving overall segmentation performance of 3D and 4D methods, but we expect that M
4
D would still perform better, because it utilizes both scans for model matching, which offers increased robustness.Our method can be expanded to handle processing of more than two lungs scans at the same time. This can be done by extending (3) and (4) accordingly. Another advantage is that M
4
D does not require any prior knowledge of the breathing state of the lungs in individual CT scans, because it does not make any assumptions about respiratory state (e.g., breathing sequence). An example for processing longitudinal lung CT scans in the context of cancer treatment planning/assessment is provided in Figure 8. Note that, due to lung disease (cancer), patient compliance with the utilized imaging protocol (e.g., acquisition at TLC) cannot be assumed.Figure 8
Comparison of results of methodsM
4
D, M
3
D, M
4
D
r
e
g
A
B, and M
4
D
r
e
g
B
A on a longitudinal CT volume pair. The time between scans A and B was 19 months, and the imaging protocol was quite different.


## 7. Conclusions

In this paper, we have presented a 4D lung segmentation approach that utilizes a new 4D robust active shape model matching method and provided an evaluation of this method on a diverse set of 76 TLC and FRC lung scan pairs. In addition, a detailed comparison with its 3D lung segmentation counterpart as well as two variants of 4D registration based lung segmentation methods was performed, demonstrating the advantages of our approach in terms of segmentation performance and/or computing time. By avoiding any assumptions about the respiratory state of the imaged lungs, our approach provides flexibility and is applicable to pairs of TLC and FRC scans, other dynamic 4D lung CT scans, and longitudinal CT studies. Thus, the developed method is suited for applications like cancer treatment planning or assessment of other lung diseases like emphysema.



---

*Source: 125648-2015-10-08.xml*