3Pa5-7
3D Modeling of Coronary Lumen Structure by IoU
Optimization in Deep Neural Network
Takashi Orihara1†, Naoya Kanno1, Hiroyuki Yagami1, Koichi Ito2, Takuro Ishii1,3,
Masanori Kawasaki4, Munenori Okubo4, Hitoshi Matsuo4, and Yoshifumi Saijo1 (1Grad.
School of Biomed. Eng., Tohoku Univ.; 2Grad. School of Info. Sci., Tohoku Univ.; 3FRIS,
Tohoku Univ.; 4Gifu Heart Center)
1. Introduction
Intravascular Ultrasound (IVUS) is a usefultechnique to evaluate the detailed vascular structurechanged by ischemic heart diseases1). On the other
hand, Fractional Flow Reserve (FFR) examination isalso used to evaluate the hemodynamics at thenarrowed coronary artery2). However, performing
both IVUS and FFR requires enormous resources
and thus is difficult in the clinical practice3).
Therefore, methods to estimate FFR from IVUSderived anatomical structures have been proposed.
To extract the anatomical structure from IVUS data,
it is essential to establish to trace the arterial wall in
the images, and deep learning models that
automatically trace the arterial wall have beenreported4,5). In such cases, noise and artifacts may
prevent proper segmentation6). In this study, wefocused on the imbalance in the number of pixels
between the vessel lumen (foreground) and otherareas (background) for segmentation of the arterial
wall from IVUS images using a deep learning model.
When performing segmentation on images with animbalance in the number of pixels in the foregroundand background, the general loss function may not
be able to correctly perform optimization for theregion of interest. Therefore, in the optimization ofneural network models, we adopted an approach that
directly optimizes Intersection over Union (IoU), astandard segmentation evaluation metric7). The
purpose of this study was to verify the effectivenessof the proposed optimization method for segmenting
the vascular lumen from IVUS images andconstructing the 3D models of the blood vessels.
2. Materials and Methods
2.1 IoU Optimization in Deep Neural Network
Many image segmentation methods using
CNN employed the cross-entropy loss function
during training to enable multi-class segmentation8).
On the other hand, our study only needed two-classsegmentation (i.e., the inside and outside of thevasculature) and thus the accuracy using IoU wasimportant. Therefore, in this study, an IoU loss
function was introduced to optimize the neuraltakashi.orihara.r6@dc.tohoku.ac.jp

network to produce the best IoU score for the trained
data. The IoU is usually defined as follows,
𝑇𝑃(1),
𝐼𝑜𝑈 =𝐹𝑃 + 𝑇𝑃 + 𝐹𝑁
where 𝑇𝑃, 𝐹𝑃, and 𝐹𝑁 refer to the number of true
positive, false positive, and false negative pixels,
respectively7). Since equation (1) is nondifferentiable and cannot define its back propagation,
we used the IoU approximation method proposed ina previous study7). Let 𝑉 = {1, 2, . . . , 𝑁} be the setof all pixels in all training dataset, 𝑋 be the output
value from the neural network and 𝑌 ∈ {0, 1}𝑉 bethe ground-truth value of the set 𝑉 , where 0represents a background pixel and 1 represents apixel in the lumen of a blood vessel. Then 𝐼𝑜𝑈
value was approximated as follows,𝐼 (𝑋)(2)
𝐼𝑜𝑈 =𝑈(𝑋)
𝐼(𝑋 ) = ∑ 𝑋𝑣 𝑌𝑣(3)𝑣∈𝑉(4).
𝑈(𝑋) = ∑𝑣∈𝑉 (𝑋𝑣 + 𝑌𝑣 − 𝑋𝑣 𝑌𝑣 )Finally, the IoU loss function was defined as follows,
𝐼 (𝑋)
(5).
𝐿𝐼𝑜𝑈 = 1 − 𝐼𝑜𝑈 = 1 −𝑈 (𝑋)2.2 Experimental Setup
The data used for training and testing theneural network consisted of 3027 IVUS images from53 subjects. The imaging was performed with a 60
MHz imaging catheter (AltaView for Terumo Corp.,Japan) and a mechanical IVUS system (VISICUBE
for Ueda Japan Radio Corp., Japan) as a part of theroutine examinations at the Gifu Heart Center. The
data collection was approved by the local ethics
committee. The ground-truth of the arterial wallboundaries in all images was determined manually
by a skilled technician. The labeled IVUS images
were divided into 2115 images (from 35 subjects) for
training and 912 images (from 18 subjects) for
testing. The training images were pre-processed bydata-augmentation techniques such as rotating andflipping to simulate a six-fold increase in the numberof training images. As the CNN model for this study,
U-Net9), a semantic segmentation model for medicalimages, was used.

Fig. 1 An example of an image in which the introduction of the IoU loss function improved thesegmentation accuracy of the lumen region of a vessel; (a) IVUS image , (b) IoU loss model (𝐼𝑜𝑈 =0.892), (c) Binary cross-entropy loss model (𝐼𝑜𝑈 = 0.482), and (d) Ground truth.

2.3 Evaluation Criteria
To verify the change in segmentation accuracy
with the introduction of the IoU loss function, wecalculated the IoU scores of the test data whentraining with the general loss function (binary-crossentropy loss function) and when using the IoU lossfunction. In addition, to verify the accuracy of the 3Dmodeling of the lumen structure of the blood vessel,
we used Root Mean Square Error (RMSE) toevaluate the error between the modeling results withthe binary-cross entropy loss function and with theIoU loss function and the true values using the lumen
regions labeled by the technicians.
3. Results and Discussion
The average IoU achieved by the trainedmodel with the IoU loss function was 0.865 ±0.106. Since the average IoU using the binary-crossentropy loss function was 0.834 ± 0.103 , thetrained model using the IoU loss function showed
improved accuracy in terms of the IoU index. Inaddition, the model using the IoU loss function
specifically produced better classification results forsome images than the model using the binary-crossentropy loss function. An example is shown in Fig.
1. This result suggests that the introduction of direct
estimation of indices for the segmentation problem
for IVUS images may lead to more robust
segmentation against artifacts present in IVUS
images (e.g., guidewires, and motion blurs).
For the 11 cases with more than 50 frames oflabeled images in the test data, the RMSE of thereconstructed lumen structures was 118.2 ± 40.5
µm for the trained model using the IoU loss function
and 110.6 ± 35.0 µm for the trained model using
the binary cross-entropy loss function. Fig. 2 shows
an example of the 3D modeling results of the vessel
lumen structure. These results show that the IoU loss
function improved the classification accuracy of theIoU index for individual images, but did not improve
the RMSE of the 3D reconstructed structures. The
reason for the lack of improvement in RMSE despite
the improvement in IoU could be that many of thecases that were not reconstructed due to insufficient

frames contained images in which the IoU wasgreatly improved by using the IoU loss function.
4. Conclusion
In this paper, we examined the effect of direct
IoU optimization of U-Net model for segmentation
the vascular lumen of the blood vessels in the IVUS
images. The results suggested that direct
optimization of the IoU metric was effective toimprove the accuracy and robustness of thesegmentation and enabled stable 3D modeling of thelesions in the coronary artery.
References
1. S. E. Nissen et al.: Circ. 103 (2001) 604.
2. N. H. J. Pijls et al.: Circ. 92 (1995) 3183.
3. W. Yu et al.: Circ. Cardiovascular Interventions
14.2 (2021) e009840.
4. F. Seike et al.: Circ. Journal (2018) CJ-17.
5. N.Takeshi et al.: Int. j. Cardiol. 333 (2021) 55.
6. Lu, Haoxuan et al.: Comput. Math. Methods Med.
2022 (2022).
7. R. M. Atiqur, and Y. Wang: Int. Symp. Vis.
Comput., Springer Cham (2016) 234.
8. S. Yusei et al.: Asian Conf. Pattern Recognit.,
Springer Cham (2019).
9. O. Ronneberger et al.: Med. Image Comput.
Comput. Assist. Interv. MICCAI 2015 9351
(2015) 234.

Fig. 2 An example of results of 3D modeling oflumen structures of blood vessels.