# Solder Joint Defect Detection in the Connectors Using Improved Faster-RCNN Algorithm
## 2 Materials and Methods
### Important Improvements
#### 2.2.3 Transfer Learning with ResNet-101  
In the target detection task, the depth of the convolutional neural network is usually up to dozens or even hundreds of layers due to the complexity of the shape, color, and other characteristics of the target in the image. However, the deeper layer is not conducive to the parameter optimization of the network, and gradient explosion or gradient disappearance are likely to occur during the training process [25]. Although batch-norm, random gradient descent and other algorithms can be used to achieve a certain degree of optimization, the effect is not obvious. Therefore, in 2015, He proposed a new network structure unit, which is called the residual unit. Meanwhile, in 2016, He improved the residual unit to make it easier to train and increase its generalization ability [26]. The remaining units can be divided into 2 layers of remaining units and 3 layers of remaining units, as shown in Figure 8. The deep convolutional network consisting of the remaining units is called ResNet.  
The residual unit changes the operation mode of the traditional convolutional network, in which, the final value obtained by convolutional layer and ReLU is shown in Equation (3).  
\[H(x)=F(x)+x \tag{3}\]  
where \(F(x)\) is the difference between the output \(H(x)\) and the input \(x\), namely the residual. In an ideal situation, when the network reaches a certain depth, if the network state is already optimal, \(F(x)\) should be set as 0, which is equivalent to the output \(H(x)\) of the residual unit as \(x\), so that the current depth network does not degrade, which ensures the accuracy of the deep network model.  
\begin{table}
\begin{tabular}{r} \hline \hline
**Ground Truth Boxes: \{(w_{i}^{*},h_{i}^{*})\}, \(i\in\)[1, N]\} \\ \hline Initialize k clustering centers: \(\left\{\left(w_{j},h_{j}\right),\ j\in[1,\ k]\right\}\) \\ Iteration: stop if the change of new clustering centers is smaller than threshold. \\   for \(i\) in range (N): \\   for \(j\) in range (\(k\)): \\  \(d_{i}(j)=1-IOU\left(\left(w_{i}^{*},h_{i}^{*}\right),\ \left(w_{j},h_{j} \right)\right)\) \\  \(cls(i)=Index(\underset{j\in[1,k]}{min}d_{i}(j))\) \\ Calculate new \(k\) clustering centers: \(w_{j}^{\prime}=\frac{1}{N_{i}}\sum w_{i}^{*}(j),\ h_{j}^{\prime}=\frac{1}{N_{ j}}\sum h_{i}^{*}(j),i\in[1,\ N],j\in[1,\ k]\) \\ \hline \hline \end{tabular}
\end{table}
Table 2: K-means clustering pseudo-code.  
Figure 7: Anchor boxes average IOU under different number of clusters. In this paper, 9 clusters are selected to realize the generation of anchor boxes, with an average IOU of 80.25%.  
ResNet has a variety of network structures, which can be divided into 18, 34, 50, 101, and 152 layers according to the convolutional layer depth. The residual units contained in different network structures are slightly different, as shown in Table 3. ResNet-18 and ResNet-34 are composed of 2-layers residual units, while ResNet-50, ResNet-101, and ResNet-152 are composed of 3-layers residual units.  
At present, ResNet is widely used due to its remarkable parameter optimization ability in training [27; 28; 29; 30]. In this paper, considering the accuracy of defect detection and actual computing ability, the convolution part of ResNet-101 was selected as the feature extraction module, and the original VGG-16 was replaced to improve the accuracy of the final detection results.