# U-Net-Based Semi-Automatic Semantic Segmentation Using Adaptive Differential Evolution
## 4 Conclusions and Future Work  
Bone semantic segmentation is essential for generating a bone model to automatically diagnose osteoporosis; however, it takes much time to generate bone borderlines, which are input data from the bone model, because it is common to write borderlines by hand. To obtain borderlines automatically from input data, a neural network model is often used, but GT images are required for learning when using a neural network model for semantic segmentation. Moreover, each handwritten borderline is often rough, so line completion and parts segmentation are necessary to obtain GT images. Therefore, we proposed a semi-automatic GT generator using a clustering method, DBSCAN, and solved the combinatorial optimization problem. Figure 7: Femur segmentation results of U-net and U-net with windowing by jDE. Further, we introduced a novel input image generator for CNN models for performance improvement. The proposed method incorporates the U-net model for segmentation, and input images of U-net are generated from DICOM data, a standard CT image format. DICOM data have various parameters related to input images, so we could obtain clear images for performance improvement of U-net if the appropriate parameters are given. These parameters have deep relationships with one another, so this paper proposed automatic windowing using jDE, a continuous real-value optimizer, because these parameters are real-value. We also defined an alternative optimization problem to reduce computational costs, instead of using U-net outputs to optimize input images. We aimed to generate a bone model for each patient, so both optimizers needed to work with a few images. U-net and jDE can work with a relatively small number of images to optimize network and DICOM parameters. We used fewer than 100 images for training. We evaluated our proposed method using two open datasets and our scanned artificial bone. Our experimental results showed that the proposed method could create GT images from roughly handwritten borderlines precisely, and we found that the time required to create training images, which used to take about 10 min per image by hand, could be significantly reduced. Moreover, the proposed method outperformed the conventional U-net and DeepLabv3 models regarding the IoU, Dice coefficient, and pairwise accuracy by incorporating the proposed input image generator using jDE. From the experimental results, we determined that DeepLabv3 showed good performance for the femur, which has a simple shape, as well as U-net and the proposed method, while U-net and the proposed method showed high performance for the artificial bone and spine. Moreover, the proposed method outperformed the U-net. Although DeepLab3 requires a large number of images to train the network, it could be considered capable of coarsely extracting simple shapes, such as femurs. By contrast, the proposed method, which also adjusts the parameters of the DICOM data, can perform better in extracting complex shapes. We verified that performance improvement was observed from segmentation images generated by the proposed method. Our main contributions are not only performance improvements but also a proposal for semi-automatic training for data generation. We believe that the proposed method will allow us to generate each patient's bone simulation for practical bone diagnosis within a reasonable time. However, there are bones with more complex shapes, such as those in the hand. For practical bone diagnoses, we should evaluate such datasets and work to improve the proposed method. The proposed method does not incorporate adjacent bone information. We hypothesize that more precise segmentation will be available if we can model the adjacent relationships, because the bone shape is continuous. Our future work includes evaluating the proposed method using more complex bone shapes and incorporating continuous bone information, in order to acquire fine bone shapes accurately. Conceptualization, K.O. and D.T. ; methodology, K.O. ; software, Y.T. ; validation, K.O., Y.T.