# Methodology for Evaluating the Generalization of ResNet
## 4 Model Set Construction
### Construction Method
We hoped to unify the batch size used for training models with different network architectures, but we were limited by computing resources, so the maximum batch size we used for training was 64. For the sake of experimental diversity, we also chose values of 32 and 16 by dividing by 2;
2. **Learning rate**: It controls the step size the model takes during optimization, with larger rates potentially causing the model to overshoot minima and smaller rates leading to slower convergence, both of which can impact the model's ability to find a good balance between bias and variance. In our previous experiments, we found that when the learning rate was \(1.25\times 10^{-3}\), the model could be trained to convergence quickly, so we enlarged and reduced the learning rate by 100 times to \(1.25\times 10^{-1}\) and \(1.25\times 10^{-5}\) respectively to affect the training process of the model. This results in a model set with diverse generalization capabilities;
3. **Optimization algorithm**: It determines the path the model takes to minimize the loss function, which can influence the convergence speed and the quality of the solution found, thereby impacting the model's capacity to learn from the training data without overfitting. We chose two of the most common and more basic optimization algorithms: SGD and Adam;
4. **Regularization coefficient**: It controls the balance between fitting the training data and maintaining model simplicity, which helps prevent overfitting and encourages the model to learn more generalizable patterns. We empirically chose regular term coefficients \(2\times 10^{-4}\) and \(5\times 10^{-4}\), and then added 0, i.e., no regular term;
5. **Model structure**: It determines the complexity and representational capacity of the model, which directly influences its capability to capture underlying patterns without overfitting to the training data. Considering the computational resources and training time, the largest model structure we chose was ResNet101, and we also chose ResNet18, ResNet34, and ResNet50;
6. **Data augmentation**: It increases the diversity of the training data, which helps the model learn more robust features that can better represent the underlying data distribution, thus improving its performance on unseen data. Data augmentation is a common way to enhance the generalization ability of a model during training, so we chose to use or not apply data augmentation for training. In the experiments in this study, data enhancement was performed by randomly adding image flipping and color enhancement;
7. **Pre-trained weights**: They provide a good starting point with learned features from a vast dataset, which can transfer useful knowledge to new tasks, thereby reducing the need to learn from scratch and potentially improving the model's performance on similar data distributions. In our experiments, we found that using ResNet pre-training weights on ImageNet significantly affects the training time and generalization ability of the model, so we chose to use or not apply the pre-training weights. Table 2 delineates the specific values assigned to each hyperparameter. For each value of hyperparameters, \(\mathbf{p}\triangleq(p_{1},\dots,p_{n})\in\Theta\), where \(\Theta\triangleq\Theta_{1}\times\dots\times\Theta_{n}\). In order to ensure that the model is trained to convergence, we established specific stopping conditions. These conditions were as follows:  
1. The training loss function value should be below a threshold, which was set at 0.1. We used the cross-entropy loss function for this evaluation. 2. The model's accuracy on the training set needs to exceed a threshold of 0.95. 3. The loss function value decreases for two consecutive batches while the test error rate increases, which indicates overfitting and signals the need to stop training. 4. The training process should not exceed 150 epochs. This condition ensures that the training can be completed within a limited time frame. For more implementation details, see Appendix A.