# Mechanical Assembly Monitoring Method Based on Semi-Supervised Semantic Segmentation
## 4 Experiments  
### Assembly Image Semantic Segmentation Dataset  
In the depth image, each pixel represents the vertical distance between the camera and the object. The depth image is not affected by the interference of environmental changes such as color temperature and illumination, so it is more suitable for industrial environments. Therefore, assembled depth images are used in the study to perform semantic segmentation. Meanwhile, this study takes the assembly process of the secondary level gear reducer as an example to verify the effectiveness of the proposed AdvSemiSeg-MA network.  
A synthesized assembly depth image dataset is established to verify the effectiveness of the proposed AdvSemiSeg-MA network on depth images. Firstly, SolidWorks is used to establish a 3D model of the reducer, and each part of the assembly model of the secondary level gear reducer is saved in the OBJ format. Then, 3 Dmax is used to import the assembly model for labeling, and the imaging model of the depth camera and the color camera is established. Finally, the camera's view is changed to shoot to obtain the depth image and label map of the reducer. In this way, the assembly image dataset of the synthesized depth image is established.  
As shown in Figure 5, this study divides the assembly process into six stages. Each stage selects 324 images from different views. In the synthesized dataset, there are a total of 1944 depth images and 9 categories. In this study, the dataset is divided into a training set and a test set at a ratio of 9:1. To verify the semi-supervised property of the algorithm proposed in this study, this study randomly selects 1/8 of the data in the training set as labeled data, and the remaining is used as the unlabeled data for training.  
RGB images are mostly used in the physical environment. Therefore, this paper uses an industrial camera to shoot real RGB images. To obtain an actual assembly RGB image dataset, the assembly was placed on the test bench, and the camera was suspended directly above the assembly to capture images of each assembly process. Then, the images were annotated at the pixel level to obtain the corresponding labels. In this way, an actual assembly RGB image dataset was established. As shown in Figure 6, in each assembly stage, 324 images were selected from different views. In the actual assembly RGB dataset, there are a total of 1620 depth images. In this study, the actual assembly RGB dataset was divided into a training set and a test set at a ratio of 9:1.  
Figure 5: Synthesized assembly depth image dataset.  
The public dataset PASCAL VOC 2012 is also used in this study to further verify the performance of the AdvSemiSeg-MA network. This dataset has a total of 21 categories. A total of 10,582 images of the dataset are included as a training set, and 1449 images are included as a testing set.  
In the training process on the synthesized assembly depth image dataset, this study uses depth images and RGB images with a size of 416 \(\times\) 416 and 512 \(\times\) 512 for training, and the processing batch of the network is set to 2. In the process of training on the PASCAL VOC 2012 dataset, this study uses images with a size of 321 \(\times\) 321 for training, and the processing batch of the network is set to 6. In the process of semi-supervised training, 1/8 of the training set is randomly selected as the labeled dataset, and the rest is used as an unlabeled dataset. When training the discriminator network, only the labeled dataset is used for training.