# Multiclass Segmentation of Concrete Surface Damages Using U-Net and DeepLabV3+
## 4 Conclusions  
Based on the results and discussions, the following conclusions can be inferred:  
1. U-Net and DeepLabV3+ were successfully reconstructed and trained for multiclass damage segmentation of concrete surfaces. Both models were implemented to differentiate background, crack, void, and spalling on concrete surfaces under complex scenarios.
2. U-Net generally performed better than DeepLabV3+. The F1 and mIoU of U-Net were 0.7199 and 0.5993, respectively, while the F1 and mIoU of DeepLabV3+ were 0.6478 and 0.5174, respectively. Overall, U-Net tended to have better fine-grained edge segmentation, while DeepLabV3+ tended to have coarser edge segmentation. All in all, both models performed well on a complex and limited dataset.
3. GDL successfully addressed the classes imbalance issue. By modifying the weight on the less important class, the F1 and IoU on the damage classes were improved. Compared to categorical cross entropy loss, which weighs every class the same, GDL was significantly better. Quantitatively, GDL scored around 10% better than categorical cross entropy in terms of F1 and mIoU, on both U-Net and DeepLabV3+. Qualitatively, categorical cross entropy often ignores classes with a smaller area, resulting in poor multiclass detection.
4. L1 and spatial dropout were used as the regularization techniques. Both techniques improved the overall model's convergence. Quantitatively, the use of either L1 or spatial dropout will improve the neural networks' convergence. However, by using both regularization techniques, both U-Net and DeepLabV3+ improved further. Qualitatively, when no regularization was used, U-Net often miscategorized black lines as cracks, and DeepLabV3+ often did not detect spalling. When only L1 or spatial dropout was used, both models improved their overall detection rate with the aforementioned misclassification seen less. When L1 and spatial dropout were used together, both models performed at their best.
5. This study provides a perspective to the deployment potential of damage inspection in infrastructure using deep learning. It is important to understand the behavior of such models in complex scenarios with limited data. For the future scope of studies, the dataset will be further expanded in terms of number of images and types of damages to ensure better generalization. A transfer-learning-based approach can also be implemented to significantly increase the model performance. For real-time detection, a new architecture and multisensor fusion can be constructed and trained using a faster GPU for inference.  
Data curation, P.N.H.; analysis, P.N.H.; writing--original draft preparation, P.N.H.; visualization, P.N.H.; methodology, P.N.H., D.S. and L.E.; supervision, D.S., L.E. and K.N.; writing--review and editing, D.S., L.E. and K.N. All authors have read and agreed to the published version of the manuscript.  
This research received no external funding.  
The data presented in this study are openly available at \"[[https://drive.google.com/file/d/1M76](https://drive.google.com/file/d/1M76) el-TsgmfsKsMKTusONNcR_QEaEzdF/view7 usp=sharing]([https://drive.google.com/file/d/1M76](https://drive.google.com/file/d/1M76) el-TsgmfsKsMKTusONNcR_QEaEzdF/view7 usp=sharing) (accessed on 11 February 2023)\" or \"[[https://github.com/HadinataPN/Multiclass-Segmentation-of-Concrete-Surface-Damages-Using-U-Net-and-DeepLabV3-](https://github.com/HadinataPN/Multiclass-Segmentation-of-Concrete-Surface-Damages-Using-U-Net-and-DeepLabV3-)]([https://github.com/[PERSON]/Multiclass-Segmentation-of-Concrete-Surface-Damages-Using-U-Net-and-DeepLabV3-](https://github.com/[PERSON]/Multiclass-Segmentation-of-Concrete-Surface-Damages-Using-U-Net-and-DeepLabV3-)) (accessed on 11 February 2023)\".  
The authors declare no conflict of interest.