# Methodology for Evaluating the Generalization of ResNet
## 2 Related Works
### Generalization Evaluation Methods Based on Modeling Prediction  
Several empirical research studies [1; 3; 5; 19; 20] have proposed methods for predicting the generalization ability of neural networks. [PERSON] et al. [1] utilized quantile statistical values outputted by specific layers of the model to construct a regression model for predicting the model's generalization ability. They constructed small-scale and simplified model sets to train and validate generalization gap predictive models, such as linear and logarithmic models. A competition named Predicting Generalization in Deep Learning (PGDL) was proposed based on this model set to predict the generalization ability of deep neural network models. This competition promoted research on using experimental methods to build models to predict the generalization ability of neural network models. [PERSON] et al. [19] treated the middle layer of the model as a classifier and selected the output data of a specific layer to establish a prediction model based on margins for the model's generalization ability, employing optimal transmission theory. [PERSON] et al. [21] predicted the generalization ability of the model using perturbation response curves, which captured how a trained model's accuracy varies with different levels of perturbation in the input data and introduced two novel measures, the Gi-score and Pal-score, inspired by economic inequality metrics, to accurately predict generalization capability. These methods require not only training data but also labeled test data. Although these methods perform well on PGDL datasets, most of them require training data and have not been verified on large-scale models. [PERSON] et al. [5] used the derivative of the loss function of the model parameters, that is, the Jacobian matrix, as a reference for evaluating the generalization ability of the model. However, this approach required significant computational resources. [PERSON] et al. [22] used the model's invariance to transformations of the input data to evaluate generalization. Then, a more efficient method for generalizability assessment was proposed which uses the normalized value of the nuclear norm of the prediction matric to evaluate the generalization ability of a model [20].  
Most of these empirical research works have been experimentally verified on simplified or small-scale model sets, which may lead to inaccurate or poor applicability of the obtained methods for evaluating model generalization. [PERSON] et al. [3] used correlation to analyze generalization evaluation methods on a large-scale model set and found that most methods have a weak correlation with the true generalization ability of the model.  
In summary, the methods for assessing the generalization ability of deep neural network models can be categorized into theoretical analysis-based approaches and empirical research methods. Theoretical analysis methods, while widely used, often have limited applicability and struggle to provide accurate upper bounds for model generalization errors. Empirical research works have proposed various modeling prediction methods, but many of them rely on modeling and training and lack validation on large-scale and complicated model sets.