# Methodology for Evaluating the Generalization of ResNet
## 5 Experimental Analysis
### Experimental Results
#### 5.2.2 Model Selection Experiments  
The generalizability evaluation method can be used for model selection. Given two models Model\({}_{1}\) and Model\({}_{2}\), the IoU values of the two models are IoU\({}_{1}^{\text{Model}}\) and IoU\({}_{2}^{\text{Model}}\). If IoU\({}_{1}^{\text{Model}}\) > IoU\({}_{2}^{\text{Model}}\), then Model\({}_{1}\) is considered to have a stronger generalization ability than Model\({}_{2}\), and accordingly, the model with a stronger generalization ability can be selected. In the model set, pairwise combinations were made, and the IoU was used to select models with stronger generalization abilities. Then, the correctness of the selection was counted.  
Table 4 presents the results of the model selection experiments, and compares the model selections using IoU within identical network structures (first four rows) and across  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Accuracy Threshold** & \(\geq\)**95** & \(\geq\)**90** & \(\geq\)**85** & \(\geq\)**80** \\ \hline \multirow{6}{*}{ResNet18} & Spectral Norm & 0.0592 & 0.0330 & 0.0429 & 0.0451 \\  & Nuclear Norm & \(-\)0.3243 & \(-\)0.2874 & \(-\)0.2793 & \(-\)0.2273 \\  & EI & \(-\)0.6577 & \(-\)0.6550 & \(-\)0.6539 & \(-\)0.6462 \\  & IoU-B & \(-\)0.8587 & \(-\)0.8556 & \(-\)0.8466 & \(-\)0.8358 \\  & IoU & \(\mathbf{-0.9144}\) & \(\mathbf{-0.9096}\) & \(\mathbf{-0.8996}\) & \(\mathbf{-0.8911}\) \\ \hline \multirow{6}{*}{ResNet34} & Spectral Norm & 0.0579 & 0.0324 & 0.0421 & 0.0424 \\  & Nuclear Norm & \(-\)0.4963 & \(-\)0.4934 & \(-\)0.4695 & \(-\)0.4432 \\  & EI & \(-\)0.7316 & \(-\)0.7339 & \(-\)0.7324 & \(-\)0.7303 \\  & IoU-B & \(-\)0.8207 & \(-\)0.8205 & \(-\)0.8139 & \(-\)0.8085 \\  & IoU & \(\mathbf{-0.8273}\) & \(\mathbf{-0.8252}\) & \(\mathbf{-0.8183}\) & \(\mathbf{-0.8149}\) \\ \hline \multirow{6}{*}{ResNet50} & Spectral Norm & 0.0574 & 0.0322 & 0.0416 & 0.0410 \\  & Nuclear Norm & \(-\)0.5610 & \(-\)0.5478 & \(-\)0.5063 & \(-\)0.4649 \\  & EI & \(-\)0.7676 & \(-\)0.7638 & \(-\)0.7642 & \(-\)0.7614 \\  & IoU-B & \(-\)0.8140 & \(\mathbf{-0.8135}\) & \(\mathbf{-0.8131}\) & \(-\)0.8040 \\  & IoU & \(\mathbf{-0.8171}\) & \(-\)0.8129 & \(-\)0.8125 & \(\mathbf{-0.8054}\) \\ \hline \multirow{6}{*}{ResNet101} & Spectral Norm & 0.0574 & 0.0323 & 0.0417 & 0.0401 \\  & Nuclear Norm & \(-\)0.5707 & \(-\)0.5599 & \(-\)0.5211 & \(-\)0.489 \\ \cline{1-1}  & EI & \(-\)0.7963 & \(-\)0.8021 & \(-\)0.8056 & \(-\)0.8076 \\ \cline{1-1}  & IoU-B & \(-\)0.8426 & \(-\)0.8417 & \(-\)0.8402 & \(-\)0.8399 \\ \cline{1-1}  & IoU & \(\mathbf{-0.8571}\) & \(\mathbf{-0.8540}\) & \(\mathbf{-0.8523}\) & \(\mathbf{-0.8520}\) \\ \hline \multirow{6}{*}{Total} & Spectral Norm & 0.0204 & 0.0198 & 0.0195 & 0.0195 \\ \cline{1-1}  & Nuclear Norm & \(-\)0.4668 & \(-\)0.4554 & \(-\)0.4357 & \(-\)0.3898 \\ \cline{1-1}  & EI & \(-\)0.7413 & \(-\)0.7419 & \(-\)0.7425 & \(-\)0.7398 \\ \cline{1-1}  & IoU-B & \(-\)0.8110 & \(-\)0.8116 & \(-\)0.8070 & \(-\)0.8005 \\ \cline{1-1}  & IoU & \(\mathbf{-0.8248}\) & \(\mathbf{-0.8228}\) & \(\mathbf{-0.8200}\) & \(\mathbf{-0.8159}\) \\ \hline \hline \end{tabular}
\end{table}
Table 3: Correlation coefficients between different valuation methods and the model generalization gap.  
different structures (last row). The high accuracy rate of the IoU-based model selection underscores the method's effectiveness and feasibility. Despite the significant impact of training data quality on model generalization, the IoU metric, calculated using the model's predicted category, remained robust even with low-quality, noisy data. This suggests that the IoU can effectively identify models with superior generalization capabilities, even when training data are suboptimal.