# Improved YOLOv5: Efficient Object Detection Using Drone Images under Various Conditions
## 2 Materials and Methods
### YOLOv5_Ours Network
This is because learning is fast and implementation is very simple due to the low amount of computation. However, the disadvantage of the ReLU activation function is that if it outputs a value less than zero, the gradient is likely to remain at zero, and the weight is likely to remain at zero forever until learning is completed. As a result, there is also a disadvantage in that learning is not conducted properly. \[\text{ReLU(x)}=\text{max(0, x)} \tag{1}\]  
\[\text{ReLU}^{\prime}(\text{x})= \begin{cases}1,&\text{if x}>0\\ 0,&\text{otherwise}\end{cases}  \tag{2}\]  
The ELU activation function is a variant of the ReLU activation function. This reduces training time and improves the test set performance of neural networks. When \(x<\) zero, the differential function is connected without breaking using the exponential function. If a broken function such as the step function is used, the loss function can be defined as uneven, resulting in local optima, as shown in Figure 2. The value of \(\alpha\) is usually specified as 1. (If \(\alpha\) is not 1, it is called SeLU.) In other words, the exclusive linear unit includes all  
Figure 1: Structure comparison of previous and current YOLOv5 (**Left** figure is previous, **Right** figure is current YOLOv5). the advantages of ReLU and solves the Dying ReLU problem. The output value is almost zero-centered, and the exp function is calculated differently from the general ReLU. \[\text{ELU}(\text{x})= \begin{cases}\text{x},&\text{if x}>0\\ \text{$\alpha$}\exp(\text{x})-1,&\text{if x}\leq 0\end{cases}  \tag{3}\]  
\[\text{ELU}^{\prime}(\text{x})= \begin{cases}1,&\text{if x}>0\\ \text{f}(\text{x})+\text{$\alpha$},&\text{if x}\leq 0\end{cases}  \tag{4}\]  
The SiLU (Swish) activation function can solve these problems, but it is only available in the hidden layers of deep neural networks and has the disadvantage that it can only be used in reinforcement learning-based systems. To solve this comprehensive problem, we used ELU (Exponential Linear Unit) as an activation function. SiLU activation function, which was previously used in the Conv layer, was replaced by the ELU activation function, as shown in Figure 3.  
\[\text{SiLU}(\text{x})=\text{x}*\frac{1}{1+\exp(-\text{x})} \tag{5}\]  
\[\text{SiLU}^{\prime}(\text{x})=\frac{1}{1+\exp(-\text{x})}+\left\{\left(\text {x}*\frac{1}{1+\exp(-\text{x})}\right)*\left(1-\frac{1}{1+\exp(-\text{x})} \right)\right\} \tag{6}\]  
Figure 3: Comparison of Conv and Conv_Ours layer (**Left** figure is the existing layer, **Right** figure is the way we suggest it). Figure 2: Graph comparison of SiLU, ELU, and ReLU activation function (**Left** figure is function, **Right** figure is a derivative function). Both the SiLU activation function and ELU activation function can solve dying RELU, but the SiLU activation function has a problem of limited use, so we replaced it with the ELU activation function. We created the Conv layer with the activation function ELU applied, and we applied this to all of the ConvELU layers in the YOLOv5_Ours structure, as shown in Figures 3-5. The formula for calculating the output size at the Conv2d layer is Equation (7). In the equation, W is the size of the input data, F is the kernel size, P is the padding size, and S is the stride. Output size of Conv2d:  
Figure 4: Comparison of original C3 and C3_Ours layer (**Left** figure is the existing layer, **Right** figure is the way we suggest it). Figure 5: Comparison of original SPPF and SPPF_Ours layer (**Above** figure is the existing layer, **Under** figure is the way we suggest it). \[\text{Output size of Conv2d}=\frac{\text{W}-\text{F}+2\text{P}}{\text{S}+1} \tag{7}\]  
The flowchart of the ConvELU layer is shown in Figure 6 as follows. BatchNorm2d layer means normalizing using average and variance, even if the data have various distributions for each batch unit in the training process. Figure 6 shows that the distribution of input values varies by batch unit or layer, but normalization makes the distribution Gaussian.