# Keypoint-Based Automated Component Placement Inspection for Printed Circuit Boards
## 1 Introduction
Efforts such as deformable DETR [25] and lite DETR [26] have been proposed to reduce computational latency. However, network sizes are not lowered. It may still be difficult to deploy such algorithms on edge devices with limited memory sizes for inference operations. For the existing object detection techniques stated above, another common drawback is that the reusability of network models is not taken into consideration. Because of supply chain management for components, even for the same products, it is likely that the corresponding PCBs are updated by the accommodation of new component classes and the removal of old ones. In such cases, it is desired that the NN models be fine-tuned for the updated PCBs. This would involve the retraining of the NN models for the detection of the new components. Nevertheless, because the NN models are shared by all the components, the retraining processes may also introduce variations in detection accuracy for the common component classes adopted by both original and updated PCBs. Therefore, robustness in inspection accuracy would be an important issue. The objective of this study is to develop novel NN models for component placement inspection on PCBs. NN models have the advantages of low labeling efforts, high object detection accuracy, low computation latency, low network sizes, and high training robustness. Corresponding Internet of Things (IoT) systems are also built for field tests. The proposed algorithm is a keypoint-based technique. Therefore, it has a simple training and inference process without the requirement of anchor boxes. The efforts required for manual labeling can be significantly lowered. In addition, no templates are necessary. The alignment issues associated with template-based techniques can then be avoided. In the proposed algorithm, the inspection of components from the same class is regarded as a single task. In the architecture, the tasks can be separated into more than one group. Different tasks in the same group may share the same network layers, termeda frontend NN, for feature extraction. Each task may have its own dedicated output layers, termed a backend NN. A novel two-stage training process is adopted for the training of the proposed network. At the first stage, only the frontend network is trained. The training procedure is intended to produce robust features [27; 28]. The resulting frontend networks serve as the initial frontend network for the training in the second stage, where both the frontend and backend networks are involved. When the network size and computational complexities of the algorithm are the important concerns, different tasks in the same group can also share the same backend NN. In these cases, the proposed algorithm is equivalent to the existing keypoint-based techniques [21], where network size may still remain low, even for a PCB consisting of large number of component classes. For applications where PCBs are constantly updated, the reuse of networks of existing components needs to be taken into consideration so that retraining efforts for the networks can be lowered. In these applications, dedicated frontend and/or backend networks can be assigned to each task. When a new component class is accommodated, training is necessary only for the backend NN for the new class and for the frontend NN for the classes sharing the same group with the new class. For the classes belonging to other groups, the corresponding networks can be directly reused. In this way, the training overhead for the inclusion of new classes can be reduced without large degradation in detection accuracy for the existing classes. The proposed NN models are deployed in edge devices of IoT systems for real-time component inspection. Although the NN models can be deployed in the cloud, data transmission congestion and delivery latency can be important issues for real-time inspection. The edge devices are able to directly process the images produced by cameras without data delivery over the Internet.