Original: W HAT ARE GAN S USEFUL FOR ?
Revised: W HAT ARE GAN S USEFUL FOR ?

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: GANs have shown how deep neural networks can be used for generative modeling, aiming at achieving the same impact that they brought for discriminative modeling.
Revised: GANs have shown how deep neural networks can be used for generative modeling, aiming at achieving the same impact that they brought for discriminative modeling.

Original: The ﬁrst results were impressive, GANs were shown to be able to generate samples in high dimensional structured spaces, like images and text, that were no copies of the training data.
Revised: The ﬁrst results were impressive, GANs were shown to be able to generate samples in high dimensional structured spaces, like images and text, that were no copies of the training data.

Original: But generative and discriminative learning are quite different.
Revised: But generative and discriminative learning are quite different.

Original: Discriminative learning has a clear end, while generative modeling is an intermediate step to understand the data or generate hypothesis.
Revised: Discriminative learning has a clear end, while generative modeling is an intermediate step to understand the data or generate hypothesis.

Original: The quality of implicit density estimation is hard to evaluate, because we cannot tell how well a data is represented by the model.
Revised: The quality of implicit density estimation is hard to evaluate, because we cannot tell how well a data is represented by the model.

Original: How can we certainly say that a generative process is generating natural images with the same distribution as we do?
Revised: How can we certainly say that a generative process is generating natural images with the same distribution as we do?

Original: In this paper, we noticed that even though GANs might not be able to generate samples from the underlying distribution (or we cannot tell at least), they are capturing some structure of the data in that high dimensional space.
Revised: In this paper, we noticed that even though GANs might not be able to generate samples from the underlying distribution (or we cannot tell at least), they are capturing some structure of the data in that high dimensional space.

Original: It is therefore needed to address how can we leverage those estimates produced by GANs in the same way we are able to use other generative modeling algorithms.
Revised: It is therefore needed to address how can we leverage those estimates produced by GANs in the same way we are able to use other generative modeling algorithms.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Generative Adversarial Models (GANs) (Goodfellow et al., 2014) or, in general, implicit generative models (Mohamed & Lakshminarayanan, 2017; Kingma & Welling, 2014) are extremely appealing to natural science because they promise to be the universal simulator.
Revised: Generative Adversarial Models (GANs) (Goodfellow et al., 2014) or, in general, implicit generative models (Mohamed & Lakshminarayanan, 2017; Kingma & Welling, 2014) are extremely appealing to natural science because they promise to be the universal simulator.

Original: If I were a genetist and had access to the genome of a few millions (billions) humans, a GAN could help me imagine any possible human.
Revised: If I were a genetist and had access to the genome of a few millions (billions) humans, a GAN could help me imagine any possible human.

Original: If I were a climate scientist and had access to global weather data, a GAN could envision any worldwide weather pattern.
Revised: If I were a climate scientist and had access to global weather data, a GAN could envision any worldwide weather pattern.

Original: These simulators are not a goal of their own, no matter how difﬁcult they are to build.
Revised: These simulators are not a goal of their own, no matter how difﬁcult they are to build.

Original: They are tools to understand genetic disease or comprehend hurricane formation or any other particularity that we are interested in.
Revised: They are tools to understand genetic disease or comprehend hurricane formation or any other particularity that we are interested in.

Original: Once, we had such a universal simulator that can generate data as nature does, we could ask any relevant question and, automatically and inexpensively have the data to validate our hypothesis.
Revised: Once, we had such a universal simulator that can generate data as nature does, we could ask any relevant question and, automatically and inexpensively have the data to validate our hypothesis.

Original: GANs have brought deep learning to generative modeling.
Revised: GANs have brought deep learning to generative modeling.

Original: Deep learning has been extremely successful in discriminative modeling, because they allow learning the features together with the classiﬁer and we do not need to rely on low-dimensional-intuitively-human-engineered features to do so.
Revised: Deep learning has been extremely successful in discriminative modeling, because they allow learning the features together with the classiﬁer and we do not need to rely on low-dimensional-intuitively-human-engineered features to do so.

Original: In (Vapnik, 1998) the author, when criticizing the paradigms created by parametric approach, declares that: “ To ﬁnd a functional dependency from data, the statistician is able to deﬁne a set of functions, linear in their parameters, that contain a good approximation to the desired function.
Revised: In (Vapnik, 1998) the author, when criticizing the paradigms created by parametric approach, declares that: “ To ﬁnd a functional dependency from data, the statistician is able to deﬁne a set of functions, linear in their parameters, that contain a good approximation to the desired function.

Original: The number of parameters describing this set is small ”.
Revised: The number of parameters describing this set is small ”.

Original: Deep Learning takes this need to deﬁne a priori this set of functions.
Revised: Deep Learning takes this need to deﬁne a priori this set of functions.

Original: Implicit models in discriminative learning make a lot of sense.
Revised: Implicit models in discriminative learning make a lot of sense.

Original: We have a clear goal in mind, like: ﬁnding objects in an image, converting speech into text or automatic machine translation, and we have a clear metric to validate our results.
Revised: We have a clear goal in mind, like: ﬁnding objects in an image, converting speech into text or automatic machine translation, and we have a clear metric to validate our results.

Original: But generative modeling, unlike discriminative learning, does not have such clear goals and metrics.
Revised: But generative modeling, unlike discriminative learning, does not have such clear goals and metrics.

Original: The generative model is always an intermediate result for solving some other problem.
Revised: The generative model is always an intermediate result for solving some other problem.

Original: We want a generative model to understand the available data and be able to generate/validate hypothesis about it.
Revised: We want a generative model to understand the available data and be able to generate/validate hypothesis about it.

Original: Broadly, we can classify generative models in two groups: those that estimate the density of the data accurately and could potentially be used for any application; and, those that extract a representation that is actionable, even though the generative model might not fully capture the data generating process, and target a speciﬁc question about the data.
Revised: Broadly, we can classify generative models in two groups: those that estimate the density of the data accurately and could potentially be used for any application; and, those that extract a representation that is actionable, even though the generative model might not fully capture the data generating process, and target a speciﬁc question about the data.

Original: In this second group, we encounter dimensionally reduction and clustering algorithms.
Revised: In this second group, we encounter dimensionally reduction and clustering algorithms.

Original: A representative example, among many others, could be Latent Dirichlet Allocation (LDA) (Blei et al., 2003).
Revised: A representative example, among many others, could be Latent Dirichlet Allocation (LDA) (Blei et al., 2003).

Original: LDA is a generative model, that allows understanding the topics in a corpus and which topics are covered in each document, but generating a document from it would be utterly incomprehensible.
Revised: LDA is a generative model, that allows understanding the topics in a corpus and which topics are covered in each document, but generating a document from it would be utterly incomprehensible.

Original: GANs clearly do not fall in this category.
Revised: GANs clearly do not fall in this category.

Original: GANs belong to the ﬁrst category, but while non-parametric models, like histograms or kernel density estimation, suffer from slow converge to the density even in the simplest low dimensional problems (Vapnik, 1998;
Revised: GANs belong to the ﬁrst category, but while non-parametric models, like histograms or kernel density estimation, suffer from slow converge to the density even in the simplest low dimensional problems (Vapnik, 1998;

Original: Lugosi & Nobel, 1996; Wasserman, 2010).
Revised: Lugosi & Nobel, 1996; Wasserman, 2010).

Original: GANs seems to be immune, because we have seen than with very limited data they have been able to generate compelling images (Radford et al., 2016; Salimans et al., 2016;
Revised: GANs seems to be immune, because we have seen than with very limited data they have been able to generate compelling images (Radford et al., 2016; Salimans et al., 2016;

Original: Gulrajani et al., 2017).
Revised: Gulrajani et al., 2017).

Original: But, is this a proper test to assure that
Revised: But, is this a proper test to assure that

Original: GANs are generating from the distribution of the data?
Revised: GANs are generating from the distribution of the data?

Original: Shouldn’t we have some test in which we can say how far we are from the true underlying distribution?
Revised: Shouldn’t we have some test in which we can say how far we are from the true underlying distribution?

Original: Shouldn’t we have a speciﬁc application in mind for which GAN might be instrumental?
Revised: Shouldn’t we have a speciﬁc application in mind for which GAN might be instrumental?

Original: Furthermore the visual tests that we use nowadays are clearly ﬂawed.
Revised: Furthermore the visual tests that we use nowadays are clearly ﬂawed.

Original: Because when we look from afar the 32-by-32 pixel images might look as bedrooms (Radford et al., 2016), but as soon as we magnify them and look at them one by one, we see that in most of them there is something off.
Revised: Because when we look from afar the 32-by-32 pixel images might look as bedrooms (Radford et al., 2016), but as soon as we magnify them and look at them one by one, we see that in most of them there is something off.

Original: For higher resolutions, the images look even more unnatural (Salimans et al., 2016).
Revised: For higher resolutions, the images look even more unnatural (Salimans et al., 2016).

Original: The same thing happens for text generation (Gulrajani et al., 2017).
Revised: The same thing happens for text generation (Gulrajani et al., 2017).

Original: Most of the published results for GANs have the following ﬂavor: GANs are unstable and we do such and such to make them easier and better to train them (Nowozin et al., 2016; Arjovsky & Bottou, 2017; Arjovsky et al., 2017; Radford et al., 2016;
Revised: Most of the published results for GANs have the following ﬂavor: GANs are unstable and we do such and such to make them easier and better to train them (Nowozin et al., 2016; Arjovsky & Bottou, 2017; Arjovsky et al., 2017; Radford et al., 2016;

Original: Salimans et al., 2016; Gulrajani et al., 2017; Li et al., 2015; Mroueh & Sercu, 2017; Tolstikhin et al., 2017; Mescheder et al., 2017; Nagarajan &
Revised: Salimans et al., 2016; Gulrajani et al., 2017; Li et al., 2015; Mroueh & Sercu, 2017; Tolstikhin et al., 2017; Mescheder et al., 2017; Nagarajan &

Original: Kolter, 2017; Roth et al., 2017; Mroueh et al., 2017) 1 .
Revised: Kolter, 2017; Roth et al., 2017; Mroueh et al., 2017) 1 .

Original: We are able to generate better-looking images or even text or something else.
Revised: We are able to generate better-looking images or even text or something else.

Original: But actually very few papers have studied if the GANs results really generate samples from the density, they just hope they do.
Revised: But actually very few papers have studied if the GANs results really generate samples from the density, they just hope they do.

Original: In a way, we are still asking what can we do for GANs, instead of asking what can GANs do for us.
Revised: In a way, we are still asking what can we do for GANs, instead of asking what can GANs do for us.

Original: On the one hand, our working hypothesis is that GANs will not be able to deliver accurate density estimates in high-dimensional structured spaces.
Revised: On the one hand, our working hypothesis is that GANs will not be able to deliver accurate density estimates in high-dimensional structured spaces.

Original: On the other hand, GANs are capturing some relevant information of that data, which is providing a simulator that can still be useful even if the density estimate is wrong.
Revised: On the other hand, GANs are capturing some relevant information of that data, which is providing a simulator that can still be useful even if the density estimate is wrong.

Original: There are some applications (Zhang et al., 2017; Hayes & Danezis, 2017;
Revised: There are some applications (Zhang et al., 2017; Hayes & Danezis, 2017;

Original: Hausman et al., 2017; Hitaj et al., 2017a) in which this is clearly the case.
Revised: Hausman et al., 2017; Hitaj et al., 2017a) in which this is clearly the case.

Original: For example, GANs for password generation (Hitaj et al., 2017b) do not need accurate density estimates, because the GAN is able to capture passowrds that previous methods could not.
Revised: For example, GANs for password generation (Hitaj et al., 2017b) do not need accurate density estimates, because the GAN is able to capture passowrds that previous methods could not.

Original: The frequencies might be off, but they are valid passwords nonetheless.
Revised: The frequencies might be off, but they are valid passwords nonetheless.

Original: GAN-like objective functions also emerge within the context of approximate variational inference with implicit distributions (Mescheder et al., 2017; Husz´ar, Ferenc, 2017).
Revised: GAN-like objective functions also emerge within the context of approximate variational inference with implicit distributions (Mescheder et al., 2017; Husz´ar, Ferenc, 2017).

Original: Many applications leverage on this kind of highly-scalable approximate inference machinery to train interpretable latent variable models in which an accurate estimate of the underlying distribution is not mandatory, see for instance (Srivastava & Sutton, 2017; Yang et al., 2017).
Revised: Many applications leverage on this kind of highly-scalable approximate inference machinery to train interpretable latent variable models in which an accurate estimate of the underlying distribution is not mandatory, see for instance (Srivastava & Sutton, 2017; Yang et al., 2017).

Original: The unanswered relevant question is when a GAN can be safely used in an application that requires some form of density estimation.
Revised: The unanswered relevant question is when a GAN can be safely used in an application that requires some form of density estimation.

Original: We expect that the answer to this application would be application dependent.
Revised: We expect that the answer to this application would be application dependent.

Original: It can take the form of a ﬂawed density estimate or rely on some statistics of the models that match those in the real data or described a interpretable model.
Revised: It can take the form of a ﬂawed density estimate or rely on some statistics of the models that match those in the real data or described a interpretable model.

Original: In this paper, to foster the discussion about where GANs should lead us, we ﬁrst examine the convergence proofs for GANs and why we believe they are incomplete or not suitable for analyzing GANs.
Revised: In this paper, to foster the discussion about where GANs should lead us, we ﬁrst examine the convergence proofs for GANs and why we believe they are incomplete or not suitable for analyzing GANs.

Original: We also revisit the papers that have look at GANs density estimate and why most of them are critical of GANs.
Revised: We also revisit the papers that have look at GANs density estimate and why most of them are critical of GANs.

Original: In the second part, we show two examples that reinforce this idea that GANs density estimates are very poor and they do not generate from the same distribution the data is coming from.
Revised: In the second part, we show two examples that reinforce this idea that GANs density estimates are very poor and they do not generate from the same distribution the data is coming from.

Original: Our experiments concur with the papers detailed in the second section of the paper.
Revised: Our experiments concur with the papers detailed in the second section of the paper.

Original:  D ENSITY E STIMATION FOR GAN S 2. P ROOFS OF CONVERGENCE
Revised:  D ENSITY E STIMATION FOR GAN S 2. P ROOFS OF CONVERGENCE

Original: There are four papers in which the convergence of the GANs to the true density is analyzed.
Revised: There are four papers in which the convergence of the GANs to the true density is analyzed.

Original: The ﬁrst proof that GANs can converge to the true density was given in the GAN original paper by
Revised: The ﬁrst proof that GANs can converge to the true density was given in the GAN original paper by

Original: Goodfellow et al.
Revised: Goodfellow et al.

Original: Recently, a new convergence proof has been reported in (Tolstikhin et al., 2017; Liu et al., 2017).
Revised: Recently, a new convergence proof has been reported in (Tolstikhin et al., 2017; Liu et al., 2017).

Original: On the contrary, in (Arora et al., 2017), the authors show that there exist equilibrium points during the training of the GAN for which the generative neural net distribution would not converge to the real distribution of the data.
Revised: On the contrary, in (Arora et al., 2017), the authors show that there exist equilibrium points during the training of the GAN for which the generative neural net distribution would not converge to the real distribution of the data.

Original: The analysis in the original paper (Goodfellow et al., 2014) proved that, if we knew the underlying density model, the equilibrium of the game would be a generator that would draw samples from the original distribution and a discriminator that would not be able to tell the difference.
Revised: The analysis in the original paper (Goodfellow et al., 2014) proved that, if we knew the underlying density model, the equilibrium of the game would be a generator that would draw samples from the original distribution and a discriminator that would not be able to tell the difference.

Original: It assumed that if the discriminator has ﬁnite (VC) dimension and the number of samples tends to inﬁnity, then the convergence will happen.
Revised: It assumed that if the discriminator has ﬁnite (VC) dimension and the number of samples tends to inﬁnity, then the convergence will happen.

Original: But this derivation has several missing steps.
Revised: But this derivation has several missing steps.

Original: First, the law of large numbers is not enough for that convergence to take place, since one will need to prove uniform convergence too (Sch¨olkopf & Smola, 2002).
Revised: First, the law of large numbers is not enough for that convergence to take place, since one will need to prove uniform convergence too (Sch¨olkopf & Smola, 2002).

Original: Second, we need to prove that the iterative procedure, between the discriminator and generator, does not get stuck in local minima in which the generator only mimics, at best, part of the distribution.
Revised: Second, we need to prove that the iterative procedure, between the discriminator and generator, does not get stuck in local minima in which the generator only mimics, at best, part of the distribution.

Original: Third, we also need to analyze the convergence when the neural network grows and so does the input dimension (e.g. noise distribution).
Revised: Third, we also need to analyze the convergence when the neural network grows and so does the input dimension (e.g. noise distribution).

Original: We need to sort out all of them, before we can claim that GANs converge to the distribution, because otherwise we might ﬁnd that we are dropping modes or not estimating the tails of the distribution correctly.
Revised: We need to sort out all of them, before we can claim that GANs converge to the distribution, because otherwise we might ﬁnd that we are dropping modes or not estimating the tails of the distribution correctly.

Original: This second problem has not even been addressed yet, as we show in the experimental section.
Revised: This second problem has not even been addressed yet, as we show in the experimental section.

Original: Finally, there is no bound for ﬁnite set of samples, so we will never know how far we are from achieving our goal.
Revised: Finally, there is no bound for ﬁnite set of samples, so we will never know how far we are from achieving our goal.

Original: In (Tolstikhin et al., 2017), the authors propose a GAN that follows a boosting procedure.
Revised: In (Tolstikhin et al., 2017), the authors propose a GAN that follows a boosting procedure.

Original: New components are added to the mixture until the original distribution is recovered and they show exponential convergence to the underlying density.
Revised: New components are added to the mixture until the original distribution is recovered and they show exponential convergence to the underlying density.

Original: Our concern with the proof is its practical applicability, as it requires that, at each step, the GAN estimated density, call it dQ , and the true underlying density of the data, call it dP d , satisfy that βdQ ≤ dP d .
Revised: Our concern with the proof is its practical applicability, as it requires that, at each step, the GAN estimated density, call it dQ , and the true underlying density of the data, call it dP d , satisfy that βdQ ≤ dP d .

Original: However, it is indeed unknown how to design a generative network that induces a density dQ that would guarantee βdQ ≤ dP d with a non-zero β when dP d is a high-dimensional structure generative process.
Revised: However, it is indeed unknown how to design a generative network that induces a density dQ that would guarantee βdQ ≤ dP d with a non-zero β when dP d is a high-dimensional structure generative process.

Original: In (Arora et al., 2017) the authors prove that for the standard metrics (e.g. Shannon-Jensen divergence and Wasserstein-like integral probability metrics), the discriminator might stop discriminating before the estimated distribution converges to the density of the data.
Revised: In (Arora et al., 2017) the authors prove that for the standard metrics (e.g. Shannon-Jensen divergence and Wasserstein-like integral probability metrics), the discriminator might stop discriminating before the estimated distribution converges to the density of the data.

Original: They also show a weaker result, in which convergence might happen, but the estimated density might be off, if the discriminator based on a deep neural network is not large enough (sufﬁcient VC dimension).
Revised: They also show a weaker result, in which convergence might happen, but the estimated density might be off, if the discriminator based on a deep neural network is not large enough (sufﬁcient VC dimension).

Original: Finally, in (Liu et al., 2017) the authors worry about two important aspects of GAN convergence: what how good the generative distribution approximates the real distribution; and, when does this convergence takes place.
Revised: Finally, in (Liu et al., 2017) the authors worry about two important aspects of GAN convergence: what how good the generative distribution approximates the real distribution; and, when does this convergence takes place.

Original: For the ﬁrst question the answer is the discriminator forces some kind of moment matching between the real and fake distributions.
Revised: For the ﬁrst question the answer is the discriminator forces some kind of moment matching between the real and fake distributions.

Original: In order to get full representation of the density we will need that the discriminator grows with the data.
Revised: In order to get full representation of the density we will need that the discriminator grows with the data.

Original: For the second question, they show a week convergence result.
Revised: For the second question, they show a week convergence result.

Original: This result is somewhat complementary to Arora et al.
Revised: This result is somewhat complementary to Arora et al.

Original: (2017), because it indicates that the discriminator complexity needs to grow indeﬁnitely to achieve convergence.
Revised: (2017), because it indicates that the discriminator complexity needs to grow indeﬁnitely to achieve convergence.

Original: The question that remains to be answer is the rate of convergence, as the moments need to be matched for complicated distributions might require large data and complex discriminators.
Revised: The question that remains to be answer is the rate of convergence, as the moments need to be matched for complicated distributions might require large data and complex discriminators.

Original: So in practice, we cannot tell if the generated distribution is close enough to the distribution we are interested in.
Revised: So in practice, we cannot tell if the generated distribution is close enough to the distribution we are interested in.

Original: Today’s GAN’s results can be explain in the light of these theoretical results.
Revised: Today’s GAN’s results can be explain in the light of these theoretical results.

Original: First, we are clearly matching some moments that relate to visual quality of natural images with ﬁnite size deep neural networks, but they might not be deep enough to capture all relevant moments and hence they will not be able to match the distribution in these images.
Revised: First, we are clearly matching some moments that relate to visual quality of natural images with ﬁnite size deep neural networks, but they might not be deep enough to capture all relevant moments and hence they will not be able to match the distribution in these images.

Original: This explains why low resolution images, in which are visual systems interpolates what it wants to see, the results are so appealing, while in large resolution the results are far from acceptable.
Revised: This explains why low resolution images, in which are visual systems interpolates what it wants to see, the results are so appealing, while in large resolution the results are far from acceptable.

Original: 2. C ONVERGENCE CRITIQUES
Revised: 2. C ONVERGENCE CRITIQUES

Original: In two recent papers (Lopez-Paz & Oquab, 2017; Sutherland et al., 2017), the authors proposed to use two-sample tests to validate GANs density estimates.
Revised: In two recent papers (Lopez-Paz & Oquab, 2017; Sutherland et al., 2017), the authors proposed to use two-sample tests to validate GANs density estimates.

Original: Both papers reach similar conclusions, if we compare images in pixel space, none of the generated distributions by GANs pass those tests.
Revised: Both papers reach similar conclusions, if we compare images in pixel space, none of the generated distributions by GANs pass those tests.

Original: Basically these methods notice that there are artefacts in which the two-sample tests can lock onto to distinguish between real and fake images.
Revised: Basically these methods notice that there are artefacts in which the two-sample tests can lock onto to distinguish between real and fake images.

Original: In the next section we replicate these experiments and ﬁnd similar conclusions.
Revised: In the next section we replicate these experiments and ﬁnd similar conclusions.

Original: In (Sutherland et al., 2017) the authors even replicated the experiment with MNIST in Salimans et al.
Revised: In (Sutherland et al., 2017) the authors even replicated the experiment with MNIST in Salimans et al.

Original: (2016) in which human could not distinguish between fake and real digits and showed that for the MMD test the difference was easy, because of the artifacts and because the GANs were only showing standard digits, but the GAN was not able to generate samples from the tail of the distribution.
Revised: (2016) in which human could not distinguish between fake and real digits and showed that for the MMD test the difference was easy, because of the artifacts and because the GANs were only showing standard digits, but the GAN was not able to generate samples from the tail of the distribution.

Original: This experiment shows the risk of using human evaluation of densities estimates, as we focus more on normalcy and under-represent the unexpected.
Revised: This experiment shows the risk of using human evaluation of densities estimates, as we focus more on normalcy and under-represent the unexpected.

Original: In the next section, we show another test that illustrates that GANs might be only focusing on the modes in the data and ignoring those tails, which might be critical in applications in which the tails carry the information of the extreme events that we might be interested in (e.g. think of weird genetic diseases or extreme weather patterns).
Revised: In the next section, we show another test that illustrates that GANs might be only focusing on the modes in the data and ignoring those tails, which might be critical in applications in which the tails carry the information of the extreme events that we might be interested in (e.g. think of weird genetic diseases or extreme weather patterns).

Original: In (Lopez-Paz & Oquab, 2017) , the authors also showed that if instead comparing in the pixel space, the comparison is made in some transformed space (in their case the ﬁnal layer in a Resnet structure), the fake samples and the true samples were indistinguishable.
Revised: In (Lopez-Paz & Oquab, 2017) , the authors also showed that if instead comparing in the pixel space, the comparison is made in some transformed space (in their case the ﬁnal layer in a Resnet structure), the fake samples and the true samples were indistinguishable.

Original: This result is quite appealing, because there are some statistics in which the GANs samples are distributed like the real samples and those statistics are sufﬁcient for our problem in hand then we might be able to rely on GANs as a simulator.
Revised: This result is quite appealing, because there are some statistics in which the GANs samples are distributed like the real samples and those statistics are sufﬁcient for our problem in hand then we might be able to rely on GANs as a simulator.

Original: The main questions is for which statistics this happens and how broad they are.
Revised: The main questions is for which statistics this happens and how broad they are.

Original: Finally, in (Arora & Zhang, 2017; Lee et al., 2017), the authors complete their theoretical paper in
Revised: Finally, in (Arora & Zhang, 2017; Lee et al., 2017), the authors complete their theoretical paper in

Original: Arora et al.
Revised: Arora et al.

Original: (2017) to show that GANs might be dropping modes of the distribution by adapting the birthday paradox for images.
Revised: (2017) to show that GANs might be dropping modes of the distribution by adapting the birthday paradox for images.

Original: Like images comes from a continuous space they need some discretization in order to show this mode dropping effect.
Revised: Like images comes from a continuous space they need some discretization in order to show this mode dropping effect.

Original: In this paper, we showed that this limitation can be also analyzed when the GAN is used to generate a discrete distribution.
Revised: In this paper, we showed that this limitation can be also analyzed when the GAN is used to generate a discrete distribution.

Original: We show that the estimated probabilities are way off
Revised: We show that the estimated probabilities are way off

Original: and we ﬁnd both mode dropping and mode enhancing to happen at the same time.
Revised: and we ﬁnd both mode dropping and mode enhancing to happen at the same time.

Original:  S IMULATIONS
Revised:  S IMULATIONS

Original: We now provide a set of experiments to support our previous discussion.
Revised: We now provide a set of experiments to support our previous discussion.

Original: We ﬁrst consider two image datasets, MNIST and CIFAR-10, and later we will revisit the recently proposed GAN-approach to enhance password guessing (PassGAN) (Hitaj et al., 2017b).
Revised: We ﬁrst consider two image datasets, MNIST and CIFAR-10, and later we will revisit the recently proposed GAN-approach to enhance password guessing (PassGAN) (Hitaj et al., 2017b).

Original: While in the ﬁrst case we have used the
Revised: While in the ﬁrst case we have used the

Original: GAN implementation proposed in (Salimans et al., 2016), PassGAN has been implemented using the improved Wasserstein GAN described in (Gulrajani et al., 2017).
Revised: GAN implementation proposed in (Salimans et al., 2016), PassGAN has been implemented using the improved Wasserstein GAN described in (Gulrajani et al., 2017).

Original: 3. S IMULATIONS USING MNIST AND CIFAR DATABASES
Revised: 3. S IMULATIONS USING MNIST AND CIFAR DATABASES

Original: We consider a speciﬁc implementation of the generator and discriminator networks, both trained over the MNIST (60,000 labeled images of handwritten numbers, plus 10,000 extra images in the test set) and CIFAR-10 datasets (50,000 labeled natural images, plus 10,000 extra images in the test set).
Revised: We consider a speciﬁc implementation of the generator and discriminator networks, both trained over the MNIST (60,000 labeled images of handwritten numbers, plus 10,000 extra images in the test set) and CIFAR-10 datasets (50,000 labeled natural images, plus 10,000 extra images in the test set).

Original: Also, we train the GAN with the ones from MNIST, i.e. 1-MNIST database, which contains 6739 training images of ones and 1134 test images.
Revised: Also, we train the GAN with the ones from MNIST, i.e. 1-MNIST database, which contains 6739 training images of ones and 1134 test images.

Original: We have reproduced the experiments and network design of (Salimans et al., 2016).
Revised: We have reproduced the experiments and network design of (Salimans et al., 2016).

Original: More speciﬁcally, the generator network is a 4 layer deep CNN, with both batch and weight normalization (Ioffe & Szegedy, 2015).
Revised: More speciﬁcally, the generator network is a 4 layer deep CNN, with both batch and weight normalization (Ioffe & Szegedy, 2015).

Original: The input z is a 100-dimensional Gaussian.
Revised: The input z is a 100-dimensional Gaussian.

Original: The discriminator network is alayer deep convolutional network with dropout, weight normalization and a Gaussian noise layer that is added to the input.
Revised: The discriminator network is alayer deep convolutional network with dropout, weight normalization and a Gaussian noise layer that is added to the input.

Original: The discriminator is trained using minibatch discrimination (MD), which helps to avoid collapse of the generator by looking at multiple data examples in combination.
Revised: The discriminator is trained using minibatch discrimination (MD), which helps to avoid collapse of the generator by looking at multiple data examples in combination.

Original: As described in (Salimans et al., 2016), within a few iterations of stochastic gradient descend (SGD), the generator network is already able to provide visually appealing images.
Revised: As described in (Salimans et al., 2016), within a few iterations of stochastic gradient descend (SGD), the generator network is already able to provide visually appealing images.

Original: In all cases, we have run 600 epochs, where per iteration all data mini-batches are processed with SGD once.
Revised: In all cases, we have run 600 epochs, where per iteration all data mini-batches are processed with SGD once.

Original: In Figure 1,
Revised: In Figure 1,

Original: [Figure]
Revised: [Figure]

Original: we illustrate samples from the generator network trained over MNIST (a) and 1-MNIST (b), and in
Revised: we illustrate samples from the generator network trained over MNIST (a) and 1-MNIST (b), and in

Original: Figure 2 samples when it is trained over CIFAR-10.
Revised: Figure 2 samples when it is trained over CIFAR-10.

Original: In most cases the samples are visually appealing, and it is likely that in most of them a person is not able to distinguish if they are real images or generated by our network.
Revised: In most cases the samples are visually appealing, and it is likely that in most of them a person is not able to distinguish if they are real images or generated by our network.

Original: These results match those in Salimans et al.
Revised: These results match those in Salimans et al.

Revised: In the next subsection we analyzed the Kernel Two-Sample Test results and we have left for the appendix the results in which we illustrate the mode dropping and the under sampling of the tails of the distribution.

Original: 3.1. K ERNEL T WO -S AMPLE T EST
Revised: 3.1. K ERNEL T WO -S AMPLE T EST

Original: In contrast to (Salimans et al., 2016), where the inception score is introduced as a metric of the quality of the images sampled from the generator network, we propose a statistical hypothesis test to study whether the distributions p ( x ) and the probability distribution of images induced by the generator network, i.e., p g ( x ) =
Revised: In contrast to (Salimans et al., 2016), where the inception score is introduced as a metric of the quality of the images sampled from the generator network, we propose a statistical hypothesis test to study whether the distributions p ( x ) and the probability distribution of images induced by the generator network, i.e., p g ( x ) =

Original: G ( z i ; θ G ) , are different.
Revised: G ( z i ; θ G ) , are different.

Original: This problem is known as the Two-Sample
Revised: This problem is known as the Two-Sample

Original: Test (Anderson et al., 1994).
Revised: Test (Anderson et al., 1994).

Original: We use the Kernel Two-Sample Test proposed in (Gretton et al., 2012), where the test statistic is the Maximum Mean Discrepancy (MMD), deﬁned as the largest difference in expectations over functions in a unit ball of a characteristic reproducing kernel Hilbert space (RKHS).
Revised: We use the Kernel Two-Sample Test proposed in (Gretton et al., 2012), where the test statistic is the Maximum Mean Discrepancy (MMD), deﬁned as the largest difference in expectations over functions in a unit ball of a characteristic reproducing kernel Hilbert space (RKHS).

Original: When MMD is large, the samples are likely from different distributions.
Revised: When MMD is large, the samples are likely from different distributions.

Original: In (Gretton et al., 2012), unbiased empirical estimators to the MMD metric are presented, and further, it is shown that the empirical MMD estimate shows a statistically signiﬁcant difference between distributions.
Revised: In (Gretton et al., 2012), unbiased empirical estimators to the MMD metric are presented, and further, it is shown that the empirical MMD estimate shows a statistically signiﬁcant difference between distributions.

Original: Given i.i.d.
Revised: Given i.i.d.

Original: samples
Revised: samples

Original: X ∼ p
Revised: X ∼ p

Original: ( x ) and ˜ X
Revised: ( x ) and ˜ X

Original: ∼ p
Revised: ∼ p

Original: g
Revised: g

Original: ( x ) , an statistical hypothesis testing is used to distinguish between the null hypothesis H 0 : p = p g and the alternative hypothesis H a : p (cid:54) =
Revised: ( x ) , an statistical hypothesis testing is used to distinguish between the null hypothesis H 0 : p = p g and the alternative hypothesis H a : p (cid:54) =

Original: p g .
Revised: p g .

Original: This is achieved by comparing the empirical MMD estimate with a particular threshold (cid:15) : if the threshold is exceeded, then the test rejects the null hypothesis.
Revised: This is achieved by comparing the empirical MMD estimate with a particular threshold (cid:15) : if the threshold is exceeded, then the test rejects the null hypothesis.

Original: Given the level α of the test (an upper bound on the Type I error), we use bootstrap resampling on the aggregated data to obtain a test threshold (cid:15) 2 .
Revised: Given the level α of the test (an upper bound on the Type I error), we use bootstrap resampling on the aggregated data to obtain a test threshold (cid:15) 2 .

Original: In our experiments we have used 1000 bootstrap shufﬂes, the radial basis function kernel with a bandwidth set as median distance between points in the aggregate sample.
Revised: In our experiments we have used 1000 bootstrap shufﬂes, the radial basis function kernel with a bandwidth set as median distance between points in the aggregate sample.

Original: Also, we ﬁx α = 0 .
Revised: Also, we ﬁx α = 0 .

Original: We use N Test samples taken at random from the MNIST and CIFAR-10 test sets and the same number of samples drawn from the generator network.
Revised: We use N Test samples taken at random from the MNIST and CIFAR-10 test sets and the same number of samples drawn from the generator network.

Original: Results are averaged over 1000 realizations.
Revised: Results are averaged over 1000 realizations.

Original: In Table 1 (a) we include the percentage of times the null hypothesis H 0 : p = p g was rejected.
Revised: In Table 1 (a) we include the percentage of times the null hypothesis H 0 : p = p g was rejected.

Original: We also include in the table the average
Revised: We also include in the table the average

Original: MMD value and two times its standard deviation, as well as the estimated threshold (cid:15) .
Revised: MMD value and two times its standard deviation, as well as the estimated threshold (cid:15) .

Original: In Table 1 (b), we randomly split the MNIST and CIFAR-10 test datasets into two disjoint sets, and we perform the same statistical test, where ideally the null hypothesis H 0 should never be rejected.
Revised: In Table 1 (b), we randomly split the MNIST and CIFAR-10 test datasets into two disjoint sets, and we perform the same statistical test, where ideally the null hypothesis H 0 should never be rejected.

Original: The comparison of results in Table 1 (a) and (b) leaves no doubt.
Revised: The comparison of results in Table 1 (a) and (b) leaves no doubt.

Original: As the number of data points increase, the empirical MMD gets further from the test threshold, strongly indicating a signiﬁcant statistical difference between distributions.
Revised: As the number of data points increase, the empirical MMD gets further from the test threshold, strongly indicating a signiﬁcant statistical difference between distributions.

Original: In contrast, the MMD metric computed in Table 1 (b) does not grow with N Test and remains very close to the threshold (cid:15) .
Revised: In contrast, the MMD metric computed in Table 1 (b) does not grow with N Test and remains very close to the threshold (cid:15) .

Original: We can safely conclude that the GAN density model does not match the distribution of the training images.
Revised: We can safely conclude that the GAN density model does not match the distribution of the training images.

Original: As mentioned before, the use of Two-Sample statistical hypothesis test to study the distribution induced by GAN has been also proposed in (Lopez-Paz & Oquab, 2017; Sutherland et al., 2017) with similar conclusions.
Revised: As mentioned before, the use of Two-Sample statistical hypothesis test to study the distribution induced by GAN has been also proposed in (Lopez-Paz & Oquab, 2017; Sutherland et al., 2017) with similar conclusions.

Original: 3.1. E XPLAINING THE GAP : N EAREST N EIGHBOR A NALYSIS B ETWEEN S AMPLE S ETS

Original: The samples generated by the GAN failed the MMD two sample test even though the generated images look as good as the original images in MNIST and CIFAR-10.

Original: We believe that the GAN fails the test because the samples are too concentrated on the modes and does not explore the full density model.

Original: This is why the samples generated for MNIST all look very good and we are almost never surprised by the weird digits that we ﬁnd sometimes in the original MNIST.

Original: We now perform a simple experiment to demonstrate the generative distribution overﬁt to the dominant modes of p ( x ) .

Original: Consider ﬁrst the 1-MNIST dataset, with 6739 training images.

Original: After jointly training the discriminative and generative networks, we sample from the generator network a set of 6739 images and, for each of them, we compute the nearest neighbor (NN) in the 1-MNIST training set.

Original: We denote by n 1-MNIST → GAN ( i ) the number of times that the i -th image in the 1-MNIST training set is the NN of images in the fake dataset generated by the GAN.

Original: Note that large n 1-MNIST → GAN ( i ) values will correspond to images from the 1-MNIST that the GAN tends to over reproduce.

Original: The n 1-MNIST → GAN ( i ) proﬁle is shown in Figure 3, where we have sorted the index images according to increasing values of n 1-MNIST → GAN ( i ) .

Original: We compare this result with the leave-one-out (LOO) NN proﬁle for the 6739 images used for training, denoted by n 1-MNIST → 1-MNIST ( i ) .

Original: This proﬁle is shown by a red dashed line in Figure 3.

Original: The results clearly indicate that there is a subset in the 1-MNIST training set that is over-reproduced in the generated artiﬁcial dataset, and thus, they correspond to modes in p g ( x ) that are not so-dominant in p ( x ) .

Original: On the other end of the spectrum the GAN represents much fewer images than the LOO NN

Original: [Table]

Original: [Figure]

Original: [Figure]

Original: proﬁle (almost 1000 samples less).

Original: This proﬁle leads us to believe that the GAN density estimation over represents the modes and under represents the tails of the distribution.

Original: In Figure 4 the n 1-MNIST → GAN ( i ) proﬁle is computed for 10 5 generated images and the two NNs are found for each image .

Original: In this case, there is still 1,300 images of ones (20% of the 1-MNIST database) that do not have a ﬁrst or second NNs in 100,000 GAN generated images.

Original: The image that

Original: [Figure]

Original: [Figure]

Original: is most popular appears as the ﬁrst or second NN in 0.4% of the cases, which seems to corroborate our initial ﬁndings that the GAN over samples the modes and under sample the tails of p ( x ) .

Original: An alternative illustration that reinforces the idea that the distribution p g ( x ) is indeed not representing the tails of p ( x ) is provided in Figure 5, where in column (a) we represent the two ones from 1-MNIST training set furthest from their LOO NNs.

Original: In (b), we represent the two images in the fake dataset sampled from the generative network (using 10 5 images) that are furthest from their LOO

Original: NNs.

Original: While in the ones from the real database, Figure 5(a), we observe images with marked calligraphic style (a reasonable unlikely one), in (b) we can appreciate images where the large distance is explained by residual noise rather than by an unusual calligraphic style.

Original: The ﬁrst column of images will never be reproduced by the GAN, while the second set are noisy regular-one images.

Original: In Figures 6 we have reproduced the same experiments for the full MNIST train set (a), where we have sample at random 60,000 images for the GAN, and the CIFAR-10 training set (b), where 50,000 images have been sampled from the GAN.

Original: Results in both cases corroborate our conclusions, showing that the distribution p g ( x ) overrepresents certain subsets of images of the training set and under represent the tails of the distribution.

Original: For the NMIST data the 1% most popular images represent 8.32% of the images in the GAN dataset and only 5.48% in the original MNIST data, while needing 6000 less images to ﬁnd the NNs for all GAN generated images.

Original: This shows that the

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: GAN covers a signiﬁcantly less fraction of the original distribution and overﬁts to the modes.

Original: For the CIFAR-10, the results are even more extreme.

Original: 3. P ASS GAN, A G ENERATIVE M ODEL FOR P ASSWORD G UESSING
Revised: 3. P ASS GAN, A G ENERATIVE M ODEL FOR P ASSWORD G UESSING

Original: PassGAN is a recent technique proposed in (Hitaj et al., 2017b), which trains a character-level GAN on leaked password datasets.
Revised: PassGAN is a recent technique proposed in (Hitaj et al., 2017b), which trains a character-level GAN on leaked password datasets.

Original: It aims at learning from the underlying training data distribution ( i.e. the distribution of leaked passwords ), autonomously understanding the characteristics that constitute a password.
Revised: It aims at learning from the underlying training data distribution ( i.e. the distribution of leaked passwords ), autonomously understanding the characteristics that constitute a password.

Original: The work is build upon the recent work of Gulrajani et al.
Revised: The work is build upon the recent work of Gulrajani et al.

Original: (2017), which shows impressive results on GAN-based text generation.
Revised: (2017), which shows impressive results on GAN-based text generation.

Original: We have reproduced the results of PassGAN to show the severe mode-dropping effect incurred, which follows the same conclusions drawn in previous experiments.
Revised: We have reproduced the results of PassGAN to show the severe mode-dropping effect incurred, which follows the same conclusions drawn in previous experiments.

Original: We also illustrate how the distribution induced by PassGAN presents modes that do not exits in the real distribution of passwords.
Revised: We also illustrate how the distribution induced by PassGAN presents modes that do not exits in the real distribution of passwords.

Original: While this clearly suggest that the density estimate is not correct, we will show that these new modes are not completely random, but one can imagine that a person would use them as a password.
Revised: While this clearly suggest that the density estimate is not correct, we will show that these new modes are not completely random, but one can imagine that a person would use them as a password.

Original: This emphasizes our idea that we still have to learn how to use the generative distribution induced by a GAN, beyond the density estimation goal.
Revised: This emphasizes our idea that we still have to learn how to use the generative distribution induced by a GAN, beyond the density estimation goal.

Original: After training PassGAN, we use sample from generative network to obtain ‘fresh’ password samples which are then used for password guessing.
Revised: After training PassGAN, we use sample from generative network to obtain ‘fresh’ password samples which are then used for password guessing.

Original: Using the same network design as in Gulrajani et al.
Revised: Using the same network design as in Gulrajani et al.

Original: (2017), we train a Wasserstein-GAN on a leaked portion of RockYou dataset (RockYou, 2010) using 23,679,744 (80%) of the data for training and the remaining 5,919,936 passwords as testing data.
Revised: (2017), we train a Wasserstein-GAN on a leaked portion of RockYou dataset (RockYou, 2010) using 23,679,744 (80%) of the data for training and the remaining 5,919,936 passwords as testing data.

Original: Out of 23,679,744 used in training 9,925,896 are unique, whereas from 5,919,936 test passwords 3,094,199 are unique.
Revised: Out of 23,679,744 used in training 9,925,896 are unique, whereas from 5,919,936 test passwords 3,094,199 are unique.

Original: The results achieved from such process are highly competitive with state-ofthe-art rule-based approaches adopted in John-the-Ripper (the Ripper, 2017) or Hashcat (HashCat, 2017) password cracking tools, refer (Hitaj et al., 2017b) for further details. 
Revised: The results achieved from such process are highly competitive with state-ofthe-art rule-based approaches adopted in John-the-Ripper (the Ripper, 2017) or Hashcat (HashCat, 2017) password cracking tools, refer (Hitaj et al., 2017b) for further details. [Table] (b)

Original: We generate 10 10 samples with PassGAN, out of which 528,834,530 (52.88%) are unique samples.
Revised: We generate 10 10 samples with PassGAN, out of which 528,834,530 (52.88%) are unique samples.

Original: PassGAN generated passwords are able to match 2,774,269 (46.9%) of the testing data.
Revised: PassGAN generated passwords are able to match 2,774,269 (46.9%) of the testing data.

Original: When checked on a new leaked passwords dataset (LinkedIn) (LinkedIn), previously unknown to PassGAN and not part of the training process, the model is able to guess 4,996,980 (11.5%) passwords out of 43,354,871 unique entries present, a remarkable result considering the fact that GAN learns on it’s own the password feats with no human intervention or manual analysis of password leaks.
Revised: When checked on a new leaked passwords dataset (LinkedIn) (LinkedIn), previously unknown to PassGAN and not part of the training process, the model is able to guess 4,996,980 (11.5%) passwords out of 43,354,871 unique entries present, a remarkable result considering the fact that GAN learns on it’s own the password feats with no human intervention or manual analysis of password leaks.

Original: In Table 2, we provide the top 30 most frequent passwords in the RockYou dataset, sorted in decreasing order, used during PassGAN training.
Revised: In Table 2, we provide the top 30 most frequent passwords in the RockYou dataset, sorted in decreasing order, used during PassGAN training.

Original: The second column provides the occurrence of a password in the dataset, whereas the third column provides the frequency of such a password in the training data.
Revised: The second column provides the occurrence of a password in the dataset, whereas the third column provides the frequency of such a password in the training data.

Original: In the fourth column, we present the GAN estimated frequency which represents the number of times a certain password was generated with respect to the total number of generations.
Revised: In the fourth column, we present the GAN estimated frequency which represents the number of times a certain password was generated with respect to the total number of generations.

Original: We notice that the generator does not follow the underlying distribution as expected, there exist password samples that are clearly underrepresented by the GAN (or even mode dropping).
Revised: We notice that the generator does not follow the underlying distribution as expected, there exist password samples that are clearly underrepresented by the GAN (or even mode dropping).

Original: See for instance ‘password’ ‘abc123’, ’babygirl’ or ’anthony’.
Revised: See for instance ‘password’ ‘abc123’, ’babygirl’ or ’anthony’.

Original: Note here, that passwords present in the training data but not generated by the GAN are shown in the table as ‘not generated’.
Revised: Note here, that passwords present in the training data but not generated by the GAN are shown in the table as ‘not generated’.

Original: The results in this table hence show a severe mode-dropping effect, because the 30th most common password with over 8,000 repetition in the over 23 million passwords is not recovered.
Revised: The results in this table hence show a severe mode-dropping effect, because the 30th most common password with over 8,000 repetition in the over 23 million passwords is not recovered.

Original: Furthermore, in Table 3, we present the top-30 passwords most frequently produced by PassGAN.
Revised: Furthermore, in Table 3, we present the top-30 passwords most frequently produced by PassGAN.

Original: Interestingly, the model produces also passwords that have low or no representation at all in the training dataset, examples of such are ‘123256’ (present 6-times in the training data), ‘iluv!u&’, which is not present at all in the training data, however generated 1,268,315 by the GAN.
Revised: Interestingly, the model produces also passwords that have low or no representation at all in the training dataset, examples of such are ‘123256’ (present 6-times in the training data), ‘iluv!u&’, which is not present at all in the training data, however generated 1,268,315 by the GAN.

Original: This shows a different effect w.r.t.
Revised: This shows a different effect w.r.t.

Original: Table 2.
Revised: Table 2.

Original: Note that PassGAN is indeed generating new modes that are somewhat structured, they are not simply sequences of random letters.
Revised: Note that PassGAN is indeed generating new modes that are somewhat structured, they are not simply sequences of random letters.

Original: Passwords like ’dangel’, ’michel’, or ’ilove’ could certainly be generated by any of us, but these passwords are not being generated at the rate at which we do.
Revised: Passwords like ’dangel’, ’michel’, or ’ilove’ could certainly be generated by any of us, but these passwords are not being generated at the rate at which we do.

Original: For these passwords that are close to common passwords there is clearly some password leakage from those most common passwords.
Revised: For these passwords that are close to common passwords there is clearly some password leakage from those most common passwords.

Original: This shows that GANs might not matching all the relevant moments, even though they are capturing relevant information
Revised: This shows that GANs might not matching all the relevant moments, even though they are capturing relevant information. [Table] michelle [Table]

Revised: anthony

Revised: [Table]

Revised:  A PPENDIX : E XPLAINING THE GAP : N EAREST N EIGHBOR A NALYSIS B ETWEEN S AMPLE S ETS

Revised: The samples generated by the GAN failed the MMD two sample test even though the generated images look as good as the original images in MNIST and CIFAR-10.

Revised: We believe that the GAN fails the test because the samples are too concentrated on the modes and does not explore the full density model.

Revised: This is why the samples generated for MNIST all look very good and we are almost never surprised by the weird digits that we ﬁnd sometimes in the original MNIST.

Revised: We now perform a simple experiment to demonstrate the generative distribution overﬁt to the dominant modes of p ( x ) .

Revised: Consider ﬁrst the 1-MNIST dataset, with 6739 training images.

Revised: After jointly training the discriminative and generative networks, we sample from the generator network a set of 6739 images and, for each of them, we compute the nearest neighbor (NN) in the 1-MNIST training set.

Revised: We denote by n 1-MNIST → GAN ( i ) the number of times that the i -th image in the 1-MNIST training set is the NN of images in the fake dataset generated by the GAN.

Revised: Note that large n 1-MNIST → GAN ( i ) values will correspond to images from the 1-MNIST that the GAN tends to over reproduce.

Revised: The n 1-MNIST → GAN ( i ) proﬁle is shown in Figure 3, where we have sorted the index images according to increasing values of n 1-MNIST → GAN ( i ) .

Revised: We compare this result with the leave-one-out (LOO) NN proﬁle for the 6739 images used for training, denoted by n 1-MNIST → 1-MNIST ( i ) .

Revised: This proﬁle is shown by a red dashed line in Figure 3.

Revised: The results clearly indicate that there is a subset in the 1-MNIST training set that is over-reproduced in the generated artiﬁcial dataset, and thus, they correspond to modes in p g ( x ) that are not so-dominant in p ( x ) .

Revised: On the other end of the spectrum the GAN represents much fewer images than the LOO NN

Revised: [Figure]

Revised: [Figure]

Revised: proﬁle (almost 1000 samples less).

Revised: This proﬁle leads us to believe that the GAN density estimation over represents the modes and under represents the tails of the distribution.

Revised: In Figure 4 the n 1-MNIST → GAN ( i ) proﬁle is computed for 10 5 generated images and the two NNs are found for each image .

Revised: In this case, there is still 1,300 images of ones (20% of the 1-MNIST database) that do not have a ﬁrst or second NNs in 100,000 GAN generated images.

Revised: The image that is most popular appears as the ﬁrst or second NN in 0.4% of the cases, which seems to corroborate our initial ﬁndings that the GAN over samples the modes and under sample the tails of p ( x ) .

Revised: An alternative illustration that reinforces the idea that the distribution p g ( x ) is indeed not representing the tails of p ( x ) is provided in Figure 5, where in column (a) we represent the two ones from 1-MNIST training set furthest from their LOO NNs.

Revised: In (b), we represent the two images in the fake dataset sampled from the generative network (using 10 5 images) that are furthest from their LOO

Revised: NNs.

Revised: While in the ones from the real database, Figure 5(a), we observe images with marked calligraphic style (a reasonable unlikely one), in (b) we can appreciate images where the large distance is explained by residual noise rather than by an unusual calligraphic style.

Revised: The ﬁrst column of images will never be reproduced by the GAN, while the second set are noisy regular-one images.

Revised: In Figures 6 we have reproduced the same experiments for the full MNIST train set (a), where we have sample at random 60,000 images for the GAN, and the CIFAR-10 training set (b), where

Revised: [Figure]

Revised: 50,000 images have been sampled from the GAN.

Revised: Results in both cases corroborate our conclusions, showing that the distribution p g ( x ) overrepresents certain subsets of images of the training set and under represent the tails of the distribution.

Revised: For the NMIST data the 1% most popular images represent 8.32% of the images in the GAN dataset and only 5.48% in the original MNIST data, while needing 6000 less images to ﬁnd the NNs for all GAN generated images.

Revised: This shows that the

Revised: GAN covers a signiﬁcantly less fraction of the original distribution and overﬁts to the modes.

Revised: For the CIFAR-10, the results are even more extreme.