# Methodology for Evaluating the Generalization of ResNet
## 4 Model Set Construction
### Model Distribution  
The construction of a high-quality dataset is fundamental to the investigation of methodologies for assessing model generalization capabilities. The distribution of these capabilities is a critical metric for evaluating the dataset's quality. A box plot, a versatile graphical tool, provides a comprehensive statistical summary of the data distribution, delineating critical aspects such as central tendency and dispersion. Specifically, it captures the median; the interquartile range (IQR), which includes the central 50% of the data; and the lower (Q1) and upper (Q3) quartiles corresponding to the 25 th and 75 th percentiles, respectively. The plot also highlights potential outliers, represented as individual points beyond the whiskers that extend to 1.5 times the IQR from the quartiles. Collectively, these elements offer a succinct yet informative overview of the variability and central tendency of the generalization gap values across the spectrum of model architectures within the established dataset. Figure 4 illustrates the distribution of generalization capabilities across various structural models within the curated dataset. The generalization gap was employed to quantify the generalization ability of each model. Upon scrutiny of Figure 4, it becomes evident that the generalization gap values exhibited a wide range, signifying considerable heterogeneity in the generalization capabilities of the constituent models. Furthermore, the consistency in the distribution of generalization capabilities across different model architectures precludes the confounding effects of structural variance on the validation of the generalization assessment methodology. Figure 5 illustrates the quantity distribution of the models across various ranges of generalization gaps. The results depicted in the figure reveal that the majority of models were concentrated within the extremes of very small or very large generalization gaps, while only a few models exhibited intermediate values. This distribution pattern can be attributed to the training termination conditions employed during the model training process, which aimed to achieve convergence. Converged models tend to demonstrate either minimal or substantial generalization gaps. A small generalization gap indicates proficient generalization to unseen data, whereas a large gap suggests overfitting, wherein the model incorporates task-irrelevant features from the training data, resulting in impressive performance on the training set but notable degradation on new data. Models displaying \begin{table}
\begin{tabular}{c c} \hline \hline
**Hyperparameter** & **Value Range** \\ \hline Batch size & \(\left\{16,\ 32,\ 64\right\}\) \\ Learning rate & \(\left\{1.25\times 10^{-1},\ 1.25\times 10^{-3},\ 1.25\times 10^{-5}\right\}\) \\ Optimization algorithm & \(\left\{\text{Adam, SGD}\right\}\) \\ Regularization coefficient & \(\left\{0,\ 2\times 10^{-4},\ 5\times 10^{-4}\right\}\) \\ Model structure & \(\left\{\text{ResNet18, ResNet34, ResNet50, ResNet101}\right\}\) \\ Data augmentation & \(\left\{\text{True, False}\right\}\) \\ Pre-trained weights & \(\left\{\text{True, False}\right\}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Hyperparameters used in model training. intermediate generalization gap values are in a transitional state, indicating that they have not yet attained the optimal balance between bias and variance. Such models may benefit from additional training. In summary, the quantity distribution of models across different generalization gap ranges, as depicted in Figure 5, signifies that the majority of models in the dataset had reached a state of convergence. This achievement serves as a foundation for the subsequent methodological investigations into assessing model generalization capability and validating the methodology's efficacy. Furthermore, this distribution validates the efficacy of the established training termination conditions, ensuring both the quality and efficiency of model training.