# Keypoint-Based Automated Component Placement Inspection for Printed Circuit Boards
## 2 Proposed Algorithm
### Component Placement Inspection for a Single Class  
Figure 1 shows the block diagram of the proposed NN model for single-class component detection. As shown in the figure, the network model can be separated into two portions: frontend network and backend network. The frontend network is used for the feature extraction of the input image. The backend network produces the results for component placement inspection. It provides a heat map indicating the likelihood that each pixel in the image belongs to the component. The size of the component can also be predicted by the backend network.  
Let \(X\) be an input image of width \(W\) and height \(H\). Our goal is to produce a heat map (\(\gamma\)) with a width of \(W/R\) and a height of \(H/R\), where \(R\) is the output stride size. Let \(Z\) be the ground truth of \(Y\). The ground truth is determined by the keypoint of each component in the input image. As shown in Figure 2, the keypoint of a component is the centroid of the component. For the sake of simplicity, assume that we only focus on the detection of components from a single class in the input image. Let \((p,q)\) be the location of a component from the class. We then compute \(Z\) for the component by splatting the keypoint of the component using a Gaussian kernel, i.e.,  
\[Z(i,j)=e^{-\frac{(i-\frac{\theta}{R})^{2}+(j-\frac{\theta}{R})^{2}}{2e^{2}}}, \tag{1}\]  
where \(Z(i,j)\) is the \((i,j)\)-th pixel of the ground truth image (\(Z\)), and \(\sigma\) is the standard deviation dependent on the size of the component. In cases in which two or more components of the class are presented, overlapping of the corresponding kernels is likely. We then take the element-wise maximum over the Gaussian kernels for components. Figure 3 shows an example of a ground truth image for the detection of capacitors on the PCB.  
Figure 1: Block diagram of the proposed NN model for single-class component detection.  
Figure 3: An example of the ground truth of a heat map for the inspection of capacitors on a PCB.  
Figure 2: An example showing the keypoint of a component. In the example, the screw in the image is the target component to be inspected. The centroid of the screw is the keypoint.  
Let \(J_{1}\) be the loss function for a heat map in the training of the proposed network model. A variant of a focal loss function [11] is adopted for the \(J_{1}\) function, i.e.,  
\[J_{1}=\sum_{i=1}^{W/R}\sum_{j=1}^{H/R}M(i,j), \tag{2}\]  
where  
\[M(i,j)=\left\{\begin{array}{ll}-(1-Y(i,j))^{\alpha}\log(Y(i,j))&,\text{if }Z(i,j)=1,\\ -(1-Z(i,j))^{\beta}(Y(i,j))^{\alpha}\log(1-Y(i,j))&,\text{otherwise},\end{array}\right. \tag{3}\]  
where \(Y(i,j)\) is the \((i,j)\)-th component of heat map \(Y\), and the \(\alpha>0\) and \(\beta>0\) parameters should be prespecified before the training.  
In addition to the detection of component location by heat map \(Y\), it may be desired to find the component size for inspection. This can be accomplished by appending a network branch in the backend network, as shown in Figure 1. Let \(S=\{S_{W},S_{H}\}\) be the output of the branch for size estimation, where \(S_{W}\) and \(S_{H}\) are the images with a width of \(W/R\) and a height of \(H/R\). Furthermore, let \(K\) be the number components and \(W_{k}\) and \(H_{k}\) be the ground truth of the width and height of the \(k\)-th component, respectively (\(k=1,\ldots,K\)). The loss function for the component sizes is defined as  
\[J_{2}=\sum_{k=1}^{K}|W_{k}-\hat{W}_{k}|+|H_{k}-\hat{H}_{k}|, \tag{4}\]  
where \(\hat{W}_{k}\) and \(\hat{H}_{k}\) are the estimated width and height of component \(k\) from \(S\). Let \((i_{k},j_{k})\) be the ground truth location of the keypoint of the \(k\)-th component (\(k=1,...,K\)). The estimation \(\hat{W}_{k}\) and \(\hat{H}_{k}\) can be computed as  
\[\hat{W}_{k}=S_{W}(i_{k},j_{k}),\ \ \ \hat{H}_{k}=S_{H}(i_{k},j_{k}). \tag{5}\]  
The total loss for the training of the proposed network model, denoted by \(J_{T}\), for a single class is then given by  
\[J_{T}=J_{1}+J_{2}. \tag{6}\]