# Synthetic Data and Computer-Vision-Based Automated Quality Inspection System for Reused Scaffolding
## 3 Materials and Methods
### Datasets  
Preparing a dataset is one of the most important and time-consuming parts of machine learning. The performance and accuracy of the future deep learning model strongly depend on its quality. Additionally, the dataset preparing step should consider such a common problem in industrial flaw detection as a class imbalance. The class imbalance arises when one class is overrepresented, and one or more is not much represented in the model training dataset. In our case, the rust detection part detected rust in the original dataset, which was prepared from real images; more than 95% of pixels were labeled as non-rusted areas and fewer than 5% were rusted. An imbalanced dataset leads to a situation where the model defines an entire image as a non-rusted image and obtains more than 90% accuracy. Several methods solve the class imbalance problem, including class weighting, data augmentation, the variable classification threshold [23], and synthetic minority over-sampling technique [24]. The original dataset was increased by using synthetic images generated and rendered by the 3D content creation suite Blender. The synthetic data can improve the performance [25] of neural networks and various machine learning algorithms by increasing the dataset and solving the class imbalance problem. In addition, the generation of synthetic data allows a reduction in the number of real data. Thus, it can be useful for reducing the time and cost of preparation. However, it is not enough to achieve only a quantitative gain from the generator according to the data; it is important that the data be of good quality. If the distribution of the generator data differs from that of the real data, the use of a mixed dataset will lead to a deterioration in the algorithm's quality. The synthetic dataset was sampled from a 3D polygonal model. All work on preparing the synthetic dataset was carried out with the Blender software package. Blender creates a realistic 3D model and adds deformations and light, and simulates different image shooting conditions, thereby making the dataset more general. Figure 2 illustrates the process flow chart for dataset generation in Blender. For model training, both the images themselves and the corresponding masks, defects, and rust were generated in the shaders stage. In addition, shaders allow rendering images and masks in different channels, saving time for labeling. The cracks were generated as a deformed ellipse and randomly added to the polygonal model. Figure 3 illustrates texture and defects generating flow. The rendered synthetic images samples are represented in Figure 4. By generating and rendering 200 synthetic images, we were able to increase the original training dataset with 300 real images, thus preventing the model from overfitting. Rendering or visualization in computer graphics is the process of translating a mathematical model of an object into a graphical representation. In Blender, this is conducted by projecting the vertices that make up the objects onto the screen plane and calculating the pixels between them by rasterizing the polygons. The real dataset consisted of 300 color images with a resolution of 1920 \(\times\) 1080, which were resized to 1024 \(\times\) 1024 for network training. The photos were taken by a scaffolding rental company, then labeled and prepared in COCO dataset format. The COCO dataset format included images and polygonal masks in JSON format. Figure 5 shows sample images with GT polygonal masks. Figure 3: Blender shading flow. Figure 2: Dataset rendering flow chart. Finally, defects were added to the polygonal model, and the point cloud was sampled, with the random noise on each point to simulate laser scanning results. Figure 6 illustrates the sampled synthetic 3D point cloud. The point cloud obtained from scanners has a noise. All sensors have specific errors that turn into various artifacts, such as isolated points that hang in the air where they should not be. In addition, the object can stand on a supporting surface.