  Generative adversarial networks (GANs) have been being widely used in various
applications. Arguably, GANs are really complex, and little has been known
about their generalization. In this paper, we make a comprehensive analysis
about generalization of GANs. We decompose the generalization error into an
explicit composition: generator error + discriminator error + optimization
error. The first two errors show the capacity of the player's families, are
irreducible and optimizer-independent. We then provide both uniform and
non-uniform generalization bounds in different scenarios, thanks to our new
bridge between Lipschitz continuity and generalization. Our bounds overcome
some major limitations of existing ones. In particular, our bounds show that
penalizing the zero- and first-order informations of the GAN loss will improve
generalization, answering the long mystery of why imposing a Lipschitz
constraint can help GANs perform better in practice. Finally, we show why data
augmentation penalizes the zero- and first-order informations of the loss,
helping the players generalize better, and hence explaining the highly
successful use of data augmentation for GANs.
