Model-based deep learning for medical image segmentation with global optimality
Creators: name: Xie, Hui
Publisher: name: University of Iowa
Year: 2022
Subjects: Deep learning
Interior Point Method
Medical Image
Min-Cut
Semantic Segmentation
public abstract
Optimization
Computer Engineering

description: Image semantic segmentation, which partitions images into multiple segments on the pixel level, plays a fundamental role in computer vision applications. With superior data representation learning capacity, deep learning (DL) methods upraise as a dominant new generation of image segmentation alternatives with remarkable performance improvements over traditional algorithms. However, a generic deep learning network does not guarantee or enforce that its solution is globally optimal in feature space. This research aims to explore integrating with deep learning the traditional optimization models with domain knowledge constraints to achieve a global optimization solution in feature space for medical image semantic segmentation.
description: This study proposed three methods to incorporate traditional continuous or combinatorial optimization models with deep learning networks: (A) integrate deep learning with the interior-point method (IPM) for constructing a multiple interactive surface model to achieve global optimality with layer order constraints in terrain surface segmentation application; (B) combine deep learning with the differentiable dynamic programming (DDP) method for building a multiple surface smoothness model to fulfill a globally optimal smooth surface segmentation in terrain surface segmentation application; (C) unify deep learning with a min-cut module through a quasi-residual connection and a creative min-cut loss to boost a globally optimal region segmentation. Experiments on public OCT (optical coherence tomography) and DFU (diabetic foot ulcer) data sets demonstrated that introducing these optimization models further improves segmentation accuracy while guaranteeing global optimality.
description: This study also presented a new data reconstructing augmentation method in severe glaucoma disease application and explored an association analysis between retinal layer thickness and arterial hypertension. The association analysis investigated the effect of increasing blood pressure on retinal layer thickness in a Chinese population of age 50+. This data reconstructing approach enriches the variety of retinal layer patterns in OCT by reconstructing A-scans at the horizontal, vertical, and jittered planes of the OCT volume image. Stacking a few local B-scans together better utilizes the 3D local contextual information. These two technologies together lead to improved segmentation accuracy and robustness.
description: This study contributed to the body of knowledge on incorporating some level of domain knowledge through traditional optimization models (interior-point method, differentiable dynamic programming, and min-cut) into the standard black-box deep learning networks in surface and region segmentation of medical images. These model-based deep learning networks can benefit from the partial domain knowledge and learning from limited data and can further improve segmentation accuracy. The association analysis found that the RNFL, GCL, and POS thicknesses were inversely correlated with higher blood pressure. In comparison, the INL thickness was positively related to higher blood pressure. These findings in retinal layer thickness may be helpful for the detection of retinal diseases.