# Application of Deep Neural Networks in Recognition of Selected Types of Objects in Digital Images
## 2 Results of Research
### GoogLeNet
The learning rate was reduced by a factor of 0.1 from its initial level of 0.001, and the minimum possible learning rate was set at 0.00001. This method allows the model to tune itself more accurately when it begins to reach the limits of its optimization capabilities. * SGD with LR reduced on Plateau: Similar to [PERSON]'s, here, the learning rate reduction mechanism was also used on Plateau, but in combination with the SGD optimizer. This allows better use of the simple mechanism for SGD to update the weights, especially in the later phases of training, where model accuracy requires finer adjustments. **LeNet-5**  
Figure 16 shows the effect of different learning strategies on the performance of the LeNet-5 model. Using the SGD optimizer with LR = 1 \(\times\) 10\({}^{-3}\) yielded the lowest accuracy of 20.27% and the highest loss of 2.21, suggesting that the model was unable to converge efficiently to optimal values. Introducing the Adam optimizer with the same learning rate improved the results, achieving an accuracy of 29.64% and a loss of 2.01. Applying the dynamic LR to Adam, where the initial LR was 1 \(\times\) 10\({}^{-3}\) with an exposure gradient, yielded even better results, with an accuracy of 30.09% and a loss of 1.98. The best results were obtained using the Adam optimizer with LR reduced on Plateau, achieving the highest accuracy of 30.36% and the lowest loss of 1.96. It was noted that the SDG optimizer with LR reduced on Plateau achieved very similar results to the regular SGD without additional modifications to the learning rate. The accuracy of both configurations was the same, at 20.27%, and the loss was also unchanged, remaining at 2.21. Moreover, the differences in training time were minimal, amounting to only about 4 s in favor of the strategy with learning rate reduction. This indicates that the optimizer did not change the learning rate setting throughout the learning period, as the model did not encounter a significant drop in loss improvement that would meet the conditions for triggering the learning rate reduction mechanism. As a result, the optimizer operated at a constant learning rate, suggesting that the reduction criteria were set too strictly for the learning rate change to be activated. Figure 17 presents GPU and memory consumption for different learning strategies. It is worth noting that GPU consumption was highest when using the Adam optimizer with LR = 1 \(\times\) 10\({}^{-3}\), at 9.32%. The lowest GPU consumption was observed using SGD, with a value of 8.72%, and SGD with Plateau reduction, with a value of 7.44%, but at the expense of poorer model accuracy. Dynamically adjusting LR in Adam resulted in relatively low GPU consumption (8.56%), while improving model accuracy and stability. Optimizing Adam with LR reduction on Plateau proved the most efficient in terms of GPU consumption, which was 8.72%, while maintaining high accuracy. During the training of the LeNet5 model, regardless of the optimization strategy used, GPU memory consumption settled  
around 3%. At the same time, RAM occupation remained at around 50%, and the average CPU usage was around 15%. **AlexNet**  
Figure 18 shows the results obtained when training the AlexNet model using different optimization strategies. The lowest accuracy of 61.16% and a loss of 1.30 were obtained using the SGD optimizer with LR = \(1\times 10^{-3}\). Using the Adam optimizer with the same learning rate led to a slight increase in accuracy to 61.79%, but the loss increased to 2.08, indicating greater model instability. The introduction of dynamic LR in Adam led to a significant increase in accuracy to 68.84% with a loss of 1.87, suggesting a better fit of the model to the data. The Adam optimizer with LR reduced on Plateau achieved the highest accuracy of 70.54% and the lowest loss of 1.29, indicating an effective fit of the model parameters. When using the SGD with LR reduced on Plateau, the accuracy increased to 63.04% and the loss dropped to 1.24, a significant improvement over the regular SGD.