# Synthetic Data and Computer-Vision-Based Automated Quality Inspection System for Reused Scaffolding
## 3 Materials and Methods
### 2D Image Segmentation  
For segmentation tasks, various solutions were used, such as Fully Connected Neural Network [26], U-net model [27], and Mask R-CNN [28]. However, Mask R-CNN is considered one of the standard CNN architectures for image instance segmentation tasks, as it not  
Figure 4: Real images with GT masks. Figure 5: Synthetic images. Figure 6: Sampled point cloud. only performs complete image classification but also segments the regions by class; that is, it creates a mask that divides the image into several classes. The original Mask R-CNN segmentation model was developed by [PERSON] [29] in 2018 as an extension of Faster R-CNN, by adding a branch for predicting an object mask in parallel with the existing branch for bounding box recognition. Mask R-CNN develops the Faster R-CNN architecture by adding another branch that predicts the position of the mask covering the found object, and thus solves the instance segmentation problem. The mask is just a rectangular matrix, in which 1 at a certain position means that the corresponding pixel belongs to an object of the specified class, and 0 means that the pixel does not belong to the object. Figure 7 illustrates the predicted heat map of the deformation area by using the Mask R-CNN network. The authors of the document conditionally divide the developed architecture into a CNN network for computing image features, which they call backbone, and a head, which is the union of parts responsible for predicting the enclosing frame, classifying an object, and determining its mask. Loss function (1) is common for them and includes three components:  
\[\text{L}=\text{L}_{\text{class}}+\text{L}_{\text{bbox}}+\text{L}_{\text{mask}} \tag{1}\]  
where \(\text{L}_{\text{class}}\) is classification loss, \(\text{L}_{\text{bbox}}\) is bounding box regression loss, and \(\text{L}_{\text{mask}}\) is mask loss  
The selection of the mask takes place in the class-agnostic style: the masks are predicted separately for each class, without prior knowledge of what is depicted in the region, and then the mask of the class that won in the independent classifier is simply selected. It is argued that this approach is more efficient than the one based on a priori knowledge of the class. One of the main modifications that arose due to the need to predict the mask is the change in the RoIPool procedure (calculating the feature matrix for the candidate region) to the so-called RoIAlign. The fact is that the feature map obtained from CNN has a smaller size than the original image, and the region covering an integer number of pixels in the image cannot be mapped into a proportional map region with an integer number of features:  
To evaluate the neural network mask prediction, this study used a loss function based on the Dice [30] coefficient, which is a measure of the overlap between the predicted mask and the ground truth mask. This coefficient ranges from 0 to 1, where 1 means complete overlap. Dice coefficient (2):  
\[\text{DICE}=2\frac{\sum\text{y}_{\text{true}}\times\text{y}_{\text{pred}}}{ \sum\text{y}_{\text{true}}+\sum\text{y}_{\text{pred}}} \tag{2}\]  
where \(\sum\) (y\({}_{\text{true}}\times\text{y}_{\text{pred}}\)) represents the common pixels between ground truth mask ytrue and predicted mask ypred. Figure 7: The predicted heat map of the deformation area. In addition, for more accurate model evaluation, this study used such metrics as precision (3) and recall (4):  
\[\text{{Precision}}=\text{TP}/(\text{TP}+\text{FP}) \tag{3}\]  
\[\text{{Recall}}=\text{TP}/(\text{TP}+\text{FN}) \tag{4}\]  
where TP true positive: prediction is positive and ground truth is positive, FP false positive: prediction is positive and ground truth is negative, FN false negative: prediction is negative and ground truth is positive.