memorize training examples one of my former interns wrote a paper his name is a Vaishnav nagarajan and he showed that it's actually hard for the generator to memorize the training data hard in a statistical learning theory sense that you can actually create reasons for why it would require quite a lot of learning steps and and a lot of observations of of different latent variables before you could memorize the training data that still doesn't really explain why when you produce samples that are new why do you get compelling