# YOLOv4-MN3 for PCB Surface Defect Detection
## 2 Methodology
### Proposed YOLOv4-MN3
#### 2.2.2 Activation Functions  
It is worth noting that few studies use YOLOv4 to detect PCB surface defect and discuss the influence of different activation functions. Previous research and initial training experiments [26] have shown that activation functions with different properties influence detection performance. Therefore, we analyze the characteristics of different activation functions and conduct comparison experiments between them to select most suitable one  
Figure 4: YOLOv4-MN3 architecture.  
Figure 3: Bneck of MobileNetV3.  
according to their training and detection performances on the customized dataset. Since MobileNetV3 has optimized its activation function, this study only selects the activation function for the neck/prediction network.  
Neural network' activation functions greatly influence their customized training convergence procedures because of their derivative, monotonicity properties, among others [27]. The selection of activation functions that perform will in training converging and detection accuracy is an essential step for model establishment [28]. Therefore, the sigmoid, tan hyperbolic (Tanh), rectified linear unit (ReLU), leaky ReLU, Swish, and Mish activation functions were implemented and configured for the training files, based on our customized dataset, to optimize the selection of an activation function.  
Figure 5 and Table 1 show the plots and expressions of the six activation functions, respectively. Sigmoid and Tanh, as traditional sigmoid-like units, have dominated neural network practice for several decades. However, they are computationally expensive, and easily lead to gradient vanishing during training. The activation functions of ReLU and Leaky ReLU are widely used in deep CNN. ReLU and Leaky ReLU are not symmetric functions and are unlike the symmetric functions of Tanh and Sigmoid. Thus, they can deal with the problem of gradient vanishing and can update weights continuously during their entire training processes [29; 30]. Leaky ReLU introduces an alpha parameter to ensure the gradient of each node would not be zero during the propagation process so that the training loss is easily be trapped into local optima. Swish is a non-monotonic and smooth activation function [31], in which the non-monotonicity property is designed to handle gradient vanishing, while its smoothness is beneficial for model generation and optimization.  
Similar to Swish, Mish is a non-monotonic and smooth function with a range of [\(\approx-0.31\),\(\infty\)). Mish outperforms other activation functions in many DL-based detectors across challenging datasets, and we can easily define a Mish activation layer in any standard DL framework for its implementation [26].  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Activation Function** & **Expression** & **Activation Function** & **Expression** \\ \hline Sigmoid & \(\mathrm{f}(x)=1/(1+e^{-x})\) & Tanh & \(\mathrm{f}(x)=\left(e^{x}-e^{-x}\right)/(e^{x}+e^{-x})\) \\ ReLU & \(\mathrm{f}(x)=\max(0,x)\) & Leaky ReLU & \(\mathrm{f}(x)=\max(ax,x)\) \\ Swish & \(\mathrm{f}(x)=x\times 1/\left(1+e^{-x}\right)\) & Mish & \(\mathrm{f}(x)=x\times\tanh(\ln(1+e^{x}))\) \\ \hline \hline \end{tabular}
\end{table}
Table 1: Expression of different activation functions.  
Figure 5: Plots of different activation function.