# Automated Multiple Concrete Damage Detection Using Instance Segmentation Deep Learning Model
## 3 Training Mask R-CNN for Multiple Concrete Damage Detection  
The Mask R-CNN used in this study is a model pretrained using a COCO dataset [34] with ResNet-101 [38] as a backbone network. After pretraining, the Mask R-CNN model was fine-tuned using images of concrete damage. Because the resolution of the mask branch is readjusted from 28 \(\times\) 28 to 56 \(\times\) 56, the weights of the mask branch were randomly initialized and the other weights were initialized using the pretrained result. To train the pretrained Mask R-CNN model for this study, 765 images in total were collected from the Internet or taken from actual structures including buildings and bridges. The number of training images was 319, 196, 249, and 237 for cracks, efflorescence, rebar exposure, and spalling, respectively. The resolutions of the images differed, ranging from 600 \(\times\) 600 to 2000 \(\times\) 2000. Photoshop, a commercial graphic software package, was used to annotate the damage on the images. Cracks, efflorescence, spalling, and rebar exposure were indicated with red, green, purple, and cyan for their respective regions, as shown in Figure 3. Please note that different from the other damage, cracks were indicated for regions including cracks and adjacent pixels, which resulted in high accuracy when quantifying the crack widths [17]. After annotation of all training images, the annotation format was converted to COCO dataset [34] using a format-converting module called Pycococcator [39] to be input to the Mask R-CNN model. The mask branch trained the annotated regions using different colors to segment the four types of damage, while the RPN, box-classification layer, and box-regression layer trained the bounding boxes marked with dotted lines. Please note that the cracks and rebar exposure with long and slender shapes were annotated using small overlapping bounding boxes to minimize the portion of intact regions in the bounding boxes, which can significantly deteriorate the performance of the model. [PERSON] et al. [11] did not explicitly mention that small bounding boxes are required to annotate cracks, but their results infer the need for small bounding boxes. [PERSON] and [PERSON] [40] mentioned this issue when detecting railways in an image. Figure 2: Resolution of mask representation comparison: (**a**) original image of concrete crack, (**b**) crack area representation by 28 \(\times\) 28 mask (in red), and (**c**) crack area representation by 56 \(\times\) 56 mask (in red). The training was performed on a high-performance computer designed for deep learning. The computer had two Intel Xeon CPUs and four GeForce GTX 1080 Ti processors with 11 GB of graphic  
Figure 3: Examples of original training images (left column) and damage annotation (right column): (**a,b**) crack, (**c,d**) rebar exposure, (**e,f**) spalling, and (**g,h**) efflorescence. memory and 64 GB of RAM. The detailed parameters in the training were as follows. The minibatch size was set as 4 to allocate single images to a GPU. To prevent the trained model from overfitting the given training data, a total of four image augmentations were randomly selected and applied to the images in each iteration. The four types of image augmentation were flipping left-right, flipping up-down, rotating 90 deg, and zero padding of any number from 0 to 100 on the four corners of the image. Each example is shown in Figure 4. The augmentations varied the location of the bounding boxes to enhance the performance of the RPN and the box-regression layer. A stochastic gradient descent enhanced by gradient clipping was used as an optimizer for training; and its learning rate, weight decay, and learning momentum were set as 0.001, 0.0001, and 0.9, respectively. All images were resized to 1024 \(\times\) 1024 considering the memory of the GPU, and the anchor sizes were set to 32, 64, 128, 256, and 512, which were the optimized sizes of the 1024 \(\times\) 1024 images. The resize aims to fit input images into the dimension of the feature-extracting convolutional network of Mask R-CNN.