# Keypoint-Based Automated Component Placement Inspection for Printed Circuit Boards
## 3 Experimental Results
### Experimental Setup  
The setup of the experiments is shown in Figure 6, which is a simple IoT inspection platform with a FLIR Blackfly S USB 3 high-resolution industrial camera. The platform, together with an edge device such as PC or Jetson Nano, can be easily integrated into a real production line for online inspection. The development of NN models is based on Keras built on the top of Tensorflow 2.0. In the experiment results, we consider only the inspection of components of screws, capacitors, mounting holes, and 3-pin chips, and 8-pin chips of the PCB, as shown in Figure 7; that is, there are \(N=5\) component classes. In many assembly lines, some of these components, such as screws, may be placed on the PCB manually. As a result, component misplacements are likely. Furthermore, it may be difficult to perform accurate component inspection for components such as 3-pin chips because of the complex background and the small component sizes. Successful inspection of the components shown in Figure 7 would be a promising indication of accurate inspection of the other PCB components. Figure 5: Examples of frontend and backend networks for a single class, where each Conv layer is a two-dimensional convolution layer and each Conv Trans layer is a two-dimensional transposed convolution layer. ResBlock is a residual block with a shortcut, BN denotes batch normalization, and RELU is an activation function. The images for our experiments have an equal size of \(512\times 512\); that is, the height and width of input images are \(W=H=512\). However, the sizes of different PCBs may vary. When PCB sizes are larger than \(512\times 512\), a sliding window operation with raster scanning order is performed over the PCB images, where the window size is \(512\times 512\). Each image acquired from the operation is an input image for the proposed algorithm; that is, in this approach, we separate the PCB images in the raster scan fashion into overlapping subimages with an equal size of \(512\times 512\) for component inspection. This provides a high degree flexibility to accommodate PCBs of different sizes in real-world scenarios. To increase the variety of the training set, different cropping results from the training PCBs are adopted as training images. Some examples are shown in Figure 8, where the objects to be detected are marked. The training set (\(\mathcal{A}\)) contains \(a=180\) images. The \(i\)-th image in the training set (\(\mathcal{A}\)) is further augmented to from a set (\(\mathcal{B}_{i}\)) containing \(b=16\) images. After cropping and augmentation of images, 2880 images (e.g, \(a\times b=2880\)) with dimensions of \(512\times 512\) are created as the augmented image set (\(\mathcal{B}\)) for training of the proposed NN model. Table 1 shows the division of set \(\mathcal{B}\) for each component class. From Figure 8, it can be observed that a single training image may contain multiple types of components. Moreover, different training images may consist of different types of components. Therefore, as shown in Table 1, the number of augmented images for each component class is different in the training set (\(\mathcal{B}\)). It can also be observed from Table 1 that although the number of augmented images for the eight-pin chip class is the lowest, there are still 1602 images for the avoidance of overfitting in training. Figure 6: The setup of the experiment. A high-resolution industrial camera is adopted for the acquisition of images from the PCBs. Figure 7: The component classes considered in the experiments: (**a**) screw, (**b**) capacitor, (**c**) mounting hole, (**d**) 3-pin chip, and (**e**) 8-pin chip. Table 2 shows the parameters in each layer of the basic NN model considered in this study.