# Application of Deep Neural Networks in Recognition of Selected Types of Objects in Digital Images
## 2 Results of Research
### Test Results
#### 2.5.1 Results of Hyperparameter Optimization Experiments for Supervised Learning Technique
(**b**) Results for batch size = 16. (**c**) Results for batch size = 32. _Optimization of the activation function_  
This section will analyze the effect of different activation functions on the performance of neural network models, using the batch size determined for the architecture. The activation functions that were tested are ReLU, Leaky ReLU, and ELU. Graphs and a tabular summary of the results show how each of these functions affects the model metrics tested. Figure 8: Comparison of GoogLeNet model training process results at different batch sizes. (**a**) Results for batch size = 8. (**b**) Results for batch size = 16. (**c**) Results for batch size = 32. Figure 9: Comparison of the results of the GoogLeNet model training process at different batch sizes, including GPU and memory consumption. (**a**) Results for batch size = 8. (**b**) Results for batch size = 16. (**c**) Results for batch size = 32. **LeNet-5**  
Figure 10 shows the results of training the LeNet-5 model using different activation functions. Using ReLU, the model achieved the lowest accuracy of 20.00% and the highest loss of 2.27. The training process took 392.49 s. In contrast, the Leaky ReLU activation function produced a significant improvement in performance, with an accuracy of 30.80% and a loss of 1.95, with a longer training time of 399 s. ELU provided stable results with an accuracy of 29.64% and a loss of 2.01, with the shortest training time of 370.13 s. Figure 11 shows the GPU and memory consumption when training the LeNet-5 model using different activation functions. Using ReLU, GPU consumption was relatively low at 7.32%. The Leaky ReLU activation function, although it yielded better results, was associated with higher GPU consumption, which was 10%. ELU, on the other hand, with a moderate GPU consumption of 8.56%, provided stable results and the shortest training time. **AlexNet**  
Figure 12 shows the results of training the AlexNet model using different activation functions. Using ReLU, the model achieved an accuracy of 63.30% and a loss of 1.55. The training process took 928.47 s. The Leaky ReLU activation function yielded an accuracy of 61.79%, but a loss of 2.08, with a training time of 889.54 s. In contrast, ELU provided an accuracy of 54.02%, with the highest loss of 2.47 and the shortest training time of 877.48 s. Figure 13 shows the GPU and memory consumption when training the AlexNet model using different activation functions. Using ReLU, the GPU consumption was 19.76%, the lowest among the activation functions tested, and the average memory consumption was 2.96%. Leaky ReLU showed a higher GPU consumption of 23.88% and the highest average memory consumption of 4.60%. In contrast, the ELU, despite a moderate GPU consumption of 23.36%, had increased average memory consumption (3.84%). Figure 10: Comparison of the results of the LeNet-5 model training process with different activation functions. (**a**) Results for ReLU. (**b**) Results for Leaky ReLU. (**c**) Results for ELU. Figure 11: Comparison of GPU and memory consumption during training of LeNet-5 model with different activation functions. (**a**) GPU and memory consumption for ReLU. (**b**) GPU and memory consumption for Leaky ReLU. (**c**) GPU and memory consumption for ELU. Figure 4:0 _Cont._