With a goal of understanding what drives generalization in deep networks, we consider several recently suggested explanations, including norm-based control, sharpness and robustness. We study how these measures can ensure generalization, highlighting the importance of scale normalization, and making a connection between sharpness and PAC-Bayes theory. We then investigate how well the measures explain different observed phenomena.
Neyshabur et al. explore several methods that could explain the generalization ability of deep neural networks, including norm-based control, sharpness, and robustness REF .
Exploring Generalization in Deep Learning
ArXiv
ArXiv
Computer Science Mathematics