  The intuition that local flatness of the loss landscape is correlated with
better generalization for deep neural networks (DNNs) has been explored for
decades, spawning many different local flatness measures. Here we argue that
these measures correlate with generalization because they are local
approximations to a global property, the volume of the set of parameters
mapping to a specific function. This global volume is equivalent to the
Bayesian prior upon initialization. For functions that give zero error on a
test set, it is directly proportional to the Bayesian posterior, making volume
a more robust and theoretically better grounded predictor of generalization
than flatness. Whilst flatness measures fail under parameter re-scaling, volume
remains invariant and therefore continues to correlate well with
generalization. Moreover, some variants of SGD can break the
flatness-generalization correlation, while the volume-generalization
correlation remains intact.
