# Synthetic Data and Computer-Vision-Based Automated Quality Inspection System for Reused Scaffolding
## 3 Materials and Methods
### 2D Image Segmentation
The mean average precision (5)--the most commonly used metric to model evaluation can be calculated as:  
\[\text{mAP}=\frac{1}{n}\sum_{k=1}^{k=n}\frac{\text{TP}_{\text{k}}}{\text{TP}_{ \text{k}}+\text{FP}_{\text{k}}} \tag{5}\]  
where \(n\) is number of classes, and k is current class. To determine better accuracy in 2D image segmentation, Mask R-CNN model was trained on the real images dataset with two different backbone architectures--ResNet-50 and ResNet-101--and then accuracy was compared with the real images dataset. The main difference between the ResNet-50 and ResNet-101 architectures is the additional 3-layer blocks (Table 2) that make model ResNet-101 deeper; however, by this increases calculation cost. Moreover, the original Mask R-CNN used the average binary cross-entropy loss for mask loss calculation, while to increase accuracy in this work Dice loss (2) function was used. Finally, Mask R-CNN was tuned by weighting loss function for better model training. Table 3 illustrates model settings and datasets. The training dataset included 300 real and 200 synthetic images, and 50 real images in the validation dataset, corresponding masks, and 50-point clouds. Since it was overfitted during 2D model training, it was decided to augment the original dataset with real images by synthetic data. The original Mask R-CNN can be used with a ResNet-50 or ResNet-101 backbone. The performances of the two backbones were compared on the real dataset. Figure 8 illustrates that the total loss of Mask R-CNN with a ResNet-101 backbone was lower than with ResNet-50; therefore, in the next experiment, a ResNet-101 backbone was used. In computer vision tasks, accuracy depends on image quality, shooting conditions, lighting brightness, reflective metal surface, shooting angle, and relative position of the camera and subject. Therefore, the original dataset with real images was augmented by synthetic images rendered in Blender to prevent overfitting and make the model more generalized. Figure 9 illustrates total loss on the validation dataset. The model was trained on real and on mixed datasets. The validation loss decreased in the mixed dataset, which \begin{table}
\begin{tabular}{c c c} \hline \hline
**Model** & & **Layers** \\ \hline ResNet-50 & & \\ \(7\times 7\), 64, stride 2 & \(3\times 3\) max pool, stride 2 & \(\begin{bmatrix}1\times\ 1,\ 64\\ 3\times\ 3,\ 64\\ 1\times\ 1,\ 256\end{bmatrix}\times 3\) & \(\begin{bmatrix}1\times\ 1,\ 128\\ 3\times\ 3,\ 128\\ 1\times\ 1,\ 512\end{bmatrix}\times 4\) & \(\begin{bmatrix}3\times\ 3,\ 256\\ 3\times\ 3,\ 256\\ 3\times\ 3,\ 256\end{bmatrix}\times 6\) & \(\begin{bmatrix}1\times\ 1,\ 512\\ 3\times\ 3,\ 512\\ 1\times\ 1,\ 2048\end{bmatrix}\times 3\) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: ResNet-50 and ResNet-101 architecture. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Backbone** & **Train Settings** & **Dataset** & **Validation** \\ \hline ResNet-50 & Original & Real images & \\ ResNet-101 & Original & Real images & \\ ResNet-101 & Dice mask loss & Mixed dataset (real + \\ ResNet-101 & Weighted + Dice loss & synthetic images) & \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Mask-RCNN model settings and datasets. indicated that the model fitted better on the mixed data. In the next experiments, only mixed data were used. The original Mask R-CNN used the average binary cross-entropy loss. In this research, Dice loss was used in the mask branch to achieve a better result. Figure 10 illustrates validation losses after the implementation of Dice loss function. Figure 8: The comparison of ResNet-50 and ResNet-101 backbone. Figure 10: Dice loss function. Figure 9: Real and mixed datasets validation losses. In addition, loss function was weighted as: rpn class loss: 1.0, rpn bbox loss: 0.3, class loss: 1.0, bbox loss: 0.8, mask loss: 0.8, which decreased total loss. Figure 11 illustrates total validation losses.