# A Multi-Step Image Pre-Enhancement Strategy for a Fish Feeding Behavior Analysis Using Efficientnet
## 3 Experiments and Discussion
### Performance of Different Training Strategies  
When analyzing and comparing the impacts of different optimizers (AdamW and SGD) combined with different loss functions (Focal Loss and Cross Entropy Loss) on neural network performance, consideration is given from two main perspectives: firstly, the combination of an optimizer and loss function on the model's fitting ability on the training set, and secondly, the effect of these combinations on the model's generalization ability on the validation set. The detailed experimental outcomes are presented in Table 2.  
Initially, when considering the use of the AdamW optimizer, its combination with Focal Loss shows only moderate performance on the training set (79.85% training accuracy) and limited generalization capability on the validation set (79.10% validation accuracy). This may indicate that the model, under this combination, struggles to effectively learn key features of the data, or that Focal Loss is not suited for this dataset. Particularly in datasets where categories are relatively balanced, using Focal Loss might not be the best choice. However, when AdamW is combined with Cross Entropy Loss, there is a significant improvement in training accuracy to 99.21%, with validation accuracy reaching 91.04%. This suggests that Cross Entropy Loss is more appropriate for model learning in this scenario. Nonetheless, higher training accuracy with a comparatively lower validation accuracy could imply an overfitting issue, where the model may have learned specific features of the training data too well, lacking generalization capability on unseen data.  
On the other hand, when using the SGD optimizer, its combination with Focal Loss performs exceptionally well on the training set (99.74% training accuracy) and also maintains high accuracy on the validation set (90.30%). This indicates that the combination of SGD with FL can effectively handle issues of class imbalance while maintaining good generalization ability. Even more notably, the combination of SGD with Cross Entropy Loss performs the best, achieving 100% training accuracy and validation accuracy of up to 97%. This combination not only achieves perfect fit on the training set but also exhibits excellent generalization ability on the validation set, suggesting that for this specific neural network and dataset, SGD combined with Cross Entropy Loss is the most effective combination.  
These results underscore the importance of choosing the right optimizer and loss function suitable for the specific dataset and problem. While AdamW is a newer optimizer that may offer advantages in handling sparse data, in some cases, the traditional SGD optimizer, when combined with the appropriate loss function, may yield better results. This indicates that even advanced optimizers and loss functions need to be adjusted and optimized for specific applications. Moreover, it also shows that the choice of optimizer and loss function can significantly affect the model's learning capability and generalization performance across different training and validation settings.