# Multiclass Segmentation of Concrete Surface Damages Using U-Net and DeepLabV3+
## 1 Introduction
Inspired from the foundation design of the FCN, U-Net [12] was introduced with a more defined encoder-decoder structure, with multiple skip connections on every convolution level. The decoder in U-Net, especially, has more channels and layers than that in FCN. It was argued that, by passing information on every level from the encoder to the decoder, localization details will be recovered [12]. Since then, U-Net has been widely used to segment objects across many disciplines [13; 14; 15]. Inspired by the encoder-decoder method, DeepLabV3 [16], which was previously a naive decoder architecture, was upgraded to DeepLabV3+ [17] with an encoder-decoder architecture. Depthwise separable convolution was introduced, which ultimately reduces computation costs on a large architecture [17]. It achieved the highest mIoU on Pascal VOC 2012 dataset [17]. In conclusion, U-Net and DeepLabV3+ have been widely used in various segmentation tasks due to their flexibility. In the application for damage detection in infrastructure, there are several research areas that focus on different structures and different types of structural changes. In the area of pavement damage detection, methods such as thresholding using the Otsu method [18], deep learning segmentation [19], and classification of damage types using a generative adversarial network [20] have been developed. In addition to methods for detecting simple visible damage such as cracks, some methods to measure the strain of structures were also developed by [PERSON] et al. [21]. This research will focus on damage detection in concrete structures. In the area of concrete structures, a block-level classification of concrete surface cracks was implemented by [PERSON] et al. [22]. To provide more detailed classification, a pixel-level method was adopted using an FCN for the binary segmentation of surface cracks [23; 24; 25]. Due to the common problem in FCNs, which is smoothened edges and coarser granularity of cracks [26], U-Net and DeepLabV3+ were implemented to tackle the problem, and both U-Net and DeepLabV3+ performed better than FCN [26] for binary crack segmentation. Some modifications were also proposed to the state-of-the-art architectures for crack detection. [PERSON] et al. [27] modified U-Net such that it requires fewer parameters to run for binary crack segmentation. [PERSON] et al. [28] implemented U-Net with a multiscale feature fusion module and Resnet block in the encoder. However, there are other types of damage on concrete surfaces such as spalling and voids. The issue with the previous research [26] is that the CNN often considers voids as cracks. Therefore, the CNN has been adapted to ignore voids that occurred on concrete surfaces [29]. Instead of just ignoring the damages other than cracks, a CNN can be adapted to differentiate the damages into different classes. [PERSON] et al. [7] have successfully implemented FCN for detecting multiple damages on concrete structures, namely efflorescence, cracks, voids, and spallings. However, their dataset mostly consists of a single type of damage in an image due to the difficulty of collecting a dataset that contains multiple damages in an image. [PERSON] et al. [30] and [PERSON] et al. [31] constructed a dataset that mostly consists of multiple damages in an image. Their dataset consists of two concrete surface damages, namely rebar exposure and delamination. Their results contributed significantly toward understanding the behavior of CNNs starting from dataset construction to model evaluation. However, most of the dataset does not deal with a highly complex foreground. To realize the implementation of such models in a complex environment, it is important to understand the boundaries of such CNN models in a highly complex foreground, especially with relatively very limited data. A previous study [26] only dealt with simple binary crack images using a publicly available dataset constructed by [PERSON] et al. [23].