Original: T EQUILA GAN: H OW TO EASILY IDENTIFY GAN SAMPLES
Revised: T EQUILA GAN: H OW TO EASILY IDENTIFY GAN SAMPLES

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: In this paper we show strategies to easily identify fake samples generated with the
Revised: In this paper we show strategies to easily identify fake samples generated with the

Original: Generative Adversarial Network framework.
Revised: Generative Adversarial Network framework.

Original: One strategy is based on the statistical analysis and comparison of raw pixel values and features extracted from them.
Revised: One strategy is based on the statistical analysis and comparison of raw pixel values and features extracted from them.

Original: The other strategy learns formal speciﬁcations from the real data and shows that fake samples violate the speciﬁcations of the real data.
Revised: The other strategy learns formal speciﬁcations from the real data and shows that fake samples violate the speciﬁcations of the real data.

Original: We show that fake samples produced with GANs have a universal signature that can be used to identify fake samples.
Revised: We show that fake samples produced with GANs have a universal signature that can be used to identify fake samples.

Original: We provide results on MNIST, CIFAR10, music and speech data.
Revised: We provide results on MNIST, CIFAR10, music and speech data.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Fake samples generated with the Generative Adversarial Networks Goodfellow et al. (2014a) framework have fooled humans and machines to believe that they are indistinguishable from real samples.
Revised: Fake samples generated with the Generative Adversarial Networks (Goodfellow et al., 2014a) framework have fooled humans and machines to believe that they are indistinguishable from real samples.

Original: Although this might be true for the naked eye and the discriminator fooled by the generator, it is unlikely that fake samples are numerically indistinguishable from real samples.
Revised: Although this might be true for the naked eye and the discriminator fooled by the generator, it is unlikely that fake samples are numerically indistinguishable from real samples.

Original: Inspired by formal methods, this paper focuses on the evaluation of fake samples with respect to statistical summaries and formal speciﬁcations computed on the real data.
Revised: Inspired by formal methods, this paper focuses on the evaluation of fake samples with respect to statistical summaries and formal speciﬁcations computed on the real data.

Original: Since the Generative Adversarial Networks paper Goodfellow et al. (2014a) in 2014, most GAN related publications use a grid of image samples to accompany theoretical and empirical results.
Revised: Since the Generative Adversarial Networks paper (Goodfellow et al., 2014a) , most GAN related publications use a grid of image samples to accompany theoretical and empirical results.

Original: Unlike Variational Autoencoders (VAEs) and other models Goodfellow et al. (2014a), most of the evaluation of the output of GAN trained Generators is qualitative: authors normally list higher sample quality as one of the advantages of their method over other methods.
Revised: Unlike Variational Autoencoders (VAEs) and other models (Goodfellow et al., 2014a), most of the evaluation of the output of GAN trained Generators is qualitative: authors normally list higher sample quality as one of the advantages of their method over other methods.

Original: Althoughsome numerical measures, like the inception score, are used to evaluate GAN samplesSalimans et al. (2016), interestingly, little is mentioned about the numerical properties of fake samples and how these properties compare to real samples.
Revised: Although numerical measures like the inception score are used to evaluate GAN samples (Salimans et al., 2016), interestingly, little is mentioned about the numerical properties of fake samples and how these properties compare to real samples.

Original: In the context of Veriﬁed Artiﬁcial IntelligenceSeshia & Sadigh (2016), it is hard to systematically verify that the output of a model satisﬁes the speciﬁcations of the data it was trained on, specially when veriﬁcation depends on the existence of perceptually meaningful features.
Revised: In the context of Veriﬁed Artiﬁcial Intelligence (Seshia & Sadigh, 2016), it is hard to systematically verify that the output of a model satisﬁes the speciﬁcations of the data it was trained on, specially when veriﬁcation depends on the existence of perceptually meaningful features.

Original: For example, consider a model that generates images of humans: although it is possible to compare color histograms of real and fake samples, we do not yet have robust algorithms to verify if an image follows speciﬁcations derived from anatomy.
Revised: For example, consider a model that generates images of humans: although it is possible to compare color histograms of real and fake samples, we do not yet have robust algorithms to verify if an image follows speciﬁcations derived from anatomy.

Original: This paper is related to the systematic veriﬁcation of fake samples and focuses on comparing numerical properties of fake and real samples.
Revised: This paper is related to the systematic veriﬁcation of fake samples and focuses on comparing numerical properties of fake and real samples.

Original: In addition to comparing statistical summaries, we investigate how the Generator approximatesstatistical modes in the real distribution and verify if the generated samples violate speciﬁcations derived from it.
Revised: In addition to comparing statistical summaries, we investigate how the Generator approximates modes in the real distribution and verify if the generated samples violate speciﬁcations derived from it.

Original: We offer the following main contributions:
Revised: We offer the following main contributions:

Original: • We show that fake samples have properties that are barely noticed with visual of samples • We show that these properties can be used to identify the source of the data (real or fake) •
Revised: • We show that fake samples have properties that are barely noticed with visual inspection • We show that these properties can be used to identify the source of the data (real or fake) •

Original: We show that fake samples violate formal speciﬁcations learned from real data
Revised: We show that fake samples violate formal speciﬁcations learned from real data

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: Despite its youth, several publications (Arjovsky & Bottou (2017), Salimans et al. (2016), Zhao et al. (2016), Radford et al. (2015)) have investigated the use of the GAN framework for sample generation and unsupervised feature learning.
Revised: Despite its youth, several publications ((Arjovsky & Bottou, 2017), (Salimans et al., 2016), (Zhao et al., 2016), (Radford et al., 2015)) have investigated the use of the GAN framework for sample generation and unsupervised feature learning.

Original: Following the procedure described in Breuleux et al. (2011) and used in Goodfellow et al. (2014a), earlier GAN papers evaluated the quality of the fake samples by ﬁtting a Gaussian Parzen window 1 to the fake samples and reporting the log-likelihood of the test set under this distribution.
Revised: Following the procedure described in (Breuleux et al., 2011) and used in (Goodfellow et al., 2014a), earlier GAN papers evaluated the quality of the fake samples by ﬁtting a Gaussian Parzen window 1 to the fake samples and reporting the log-likelihood of the test set under this distribution.

Original: As mentioned in Goodfellow et al. (2014a) , this method has some drawbacks, including its high variance and bad performance in high dimensional spaces.
Revised: As mentioned in (Goodfellow et al., 2014a), this method has some drawbacks, including its high variance and bad performance in high dimensional spaces.

Original: The inception score is another widely adopted evaluation metric that fails to provide systematic guidance on the evaluation of GAN modelsBarratt & Sharma (2018).
Revised: The inception score is another widely adopted evaluation metric that fails to provide systematic guidance on the evaluation of GAN models(Barratt & Sharma, 2018).

Original: Unlike other optimization problems, where analysis of the empirical risk is a strong indicator of progress, in GANs the decrease in loss is not always correlated with increase in image quality Arjovsky et al. (2017), and thus authors still rely on visual inspection of generated images.
Revised: Unlike other optimization problems, where analysis of the empirical risk is a strong indicator of progress, in GANs the decrease in loss is not always correlated with increase in image quality (Arjovsky et al., 2017), and thus authors still rely on visual inspection of generated images.

Original: Based on visual inspection, authors conﬁrm that they have not observed mode collapse or that their framework is robust to mode collapse if some criteria is met (Arjovsky et al. (2017), Gulrajani et al. (2017), Mao et al. (2016), Radford et al.
Revised: Based on visual inspection, authors conﬁrm that they have not observed mode collapse or that their framework is robust to mode collapse if some criteria is met ((Arjovsky et al., 2017), (Gulrajani et al., 2017), (Mao et al., 2016), (Radford et al., 2015)).

Original: In practice, github issues where practitioners report mode collapse or not enough variety abound.
Revised: In practice, github issues where practitioners report mode collapse or not enough variety abound.

Original: In their publications, Mao et al. (2016), Arjovsky et al. (2017) and Gulrajani et al. (2017) propose alternative objective functions and algorithms that circumvent problems that are common when using the original GAN objective described in Goodfellow et al.
Revised: In their publications, (Mao et al., 2016), (Arjovsky et al., 2017) and (Gulrajani et al., 2017) propose alternative objective functions and algorithms that circumvent problems that are common when using the original GAN objective described in (Goodfellow et al., 2014a).

Original: (2014a). The problems addressed include instability of learning, mode collapse and meaningful loss curves Salimans et al.
Revised: The problems addressed include instability of learning, mode collapse and meaningful loss curves (Salimans et al., 2016).

Original: These alternatives do not eliminate the need or excitement 2 of visually inspecting GAN samples during training, nor do they provide quantitative information about the generated samples.
Revised: These alternatives do not eliminate the need, or excitement 2 , to visually inspect GAN samples during training, nor do they provide quantitative information about the generated samples.

Original:  M ETHODOLOGY
Revised:  M ETHODOLOGY

Original: The experiments in this paper focus on three points: the ﬁrst shows that fake samples have properties that are hardly noticed with visual inspection andthat are tightly related to the requirements of differentiability; the second shows that there are numerical differences between statistical moments computed on features extracted from real and fake samples that can be used to identify the data; the third shows that fake samples violate formal speciﬁcations learned from the real data.
Revised: The experiments in this paper focus on three points: the ﬁrst shows that fake samples have properties that are hardly noticed with visual inspection and tightly related to the requirements of differentiability; the second shows that there are numerical differences between statistical moments computed on features extracted from real and fake samples that can be used to identify the data; the third shows that fake samples violate formal speciﬁcations learned from the real data.

Original: In the following subsections we enumerate the datasets, features and GAN frameworks herein used.
Revised: In the following subsections we enumerate the datasets, features and GAN frameworks herein used.

Original: 3. D ATASETS
Revised: 3. D ATASETS

Original: In our experiments, we use MNIST, CIFAR10, a MIDI dataset of 389 Bach Chorales downloaded from the web and a subsample of the NIST 2004 speech dataset used in Cai et al.
Revised: In our experiments, we use MNIST, CIFAR10, a MIDI dataset of 389 Bach Chorales downloaded from the web and a subsample of the NIST 2004 speech dataset used in (Cai et al., 2018).

Original: 3. F EATURES
Revised: 3. F EATURES

Original: The spectral centroid Peeters (2004) is a feature commonly used in the audio domain, where it represents the barycenter of the spectrum.
Revised: The spectral centroid (Peeters, 2004) is a feature commonly used in the audio domain, where it represents the barycenter of the spectrum.

Original: This feature can be applied to other domains and we invite the reader to visualize Figure 12 for examples on MNIST and Mel-Spectrograms Peeters (2004).
Revised: This feature can be applied to other domains and we invite the reader to visualize Figure 12 for examples on MNIST and Mel-Spectrograms (Peeters, 2004).

Original: For each column in an image, we transform the pixel values into row probabilities by normalizing them by the column sum, after which we take the expected row value, thus obtaining the spectral centroid.
Revised: For each column in an image, we transform the pixel values into row probabilities by normalizing them by the column sum, after which we take the expected row value, thus obtaining the spectral centroid.

Original: The spectral slope adapted from Peeters (2004) is computed by applying linear regression using an overlapping sliding window of size 7.
Revised: The spectral slope adapted from (Peeters, 2004) is computed by applying linear regression using an overlapping sliding window of size 7.

Original: For each window, we regress the spectral centroids on the column number mod the window size.
Revised: For each window, we regress the spectral centroids on the column number mod the window size.

Original: Figure 13 shows these features computed on MNIST and
Revised: Figure 13 shows these features computed on MNIST and

Original: Mel-Spectrograms.
Revised: Mel-Spectrograms.

Original: 3. GAN F RAMEWORKS
Revised: 3. GAN F RAMEWORKS

Original: We investigate samples produced with the DCGAN architecture using the Least-Squares GAN (LSGAN) Mao et al. and the improved Wasserstein GAN (IWGAN/WGAN-GP) Gulrajani et al.
Revised: We investigate samples produced with the DCGAN architecture using the Least-Squares GAN (LSGAN) (Mao et al., 2016) and the improved Wasserstein GAN (IWGAN/WGAN-GP) (Gulrajani et al., 2017).

Original: We also compare adversarial MNIST samples produced with the fast gradient sign method (FGSM)
Revised: We also compare adversarial MNIST samples produced with the fast gradient sign method (FGSM)

Original: Goodfellow et al. (2014b).
Revised: (Goodfellow et al., 2014b).

Original: We evaluate the normally used non-linearities, sigmoid and tanh, on the output of the generator andand other transformations such as the scaled tanh and identity.
Revised: We evaluate the normally used non-linearities, sigmoid and tanh, on the output of the generator and other transformations such as the scaled tanh and identity.

Original:  E XPERIMENTS 4. MNIST
Revised:  E XPERIMENTS 4. MNIST

Original: The experiment focuses on showing numerical properties of fake MNIST samples and features therein, unknown to the naked eye, that can be used to identify them as produced by a GAN.
Revised: This experiment focuses on showing numerical properties of fake MNIST samples and features therein, unknown to the naked eye, that can be used to identify them as produced by a GAN.

Original: We start by comparing the distribution of features computed over the MNIST training set to other datasets, including the MNIST test set, samples generated with GANs and adversarial samples computed using the FGSM.
Revised: We start by comparing the distribution of features computed over the MNIST training set to other datasets, including the MNIST test set, samples generated with GANs and adversarial samples computed using the FGSM.

Original: The training data is scaled to [0 , 1] and the random baseline is sampled from a Bernoulli distribution with probability equal to the mean value of pixel intensities in the
Revised: The training data is scaled to [0 , 1] and the random baseline is sampled from a Bernoulli distribution with probability equal to the mean value of pixel intensities in the

Original: MNIST training data, 0.13.
Revised: MNIST training data, 0.13.

Original: Each GAN model is trained until the loss plateaus and the generated samples look similar to the real samples.
Revised: Each GAN model is trained until the loss plateaus and the generated samples look similar to the real samples.

Original: The datasets compared have 10 thousand samples each.
Revised: The datasets compared have 10 thousand samples each.

Original: Visual inspection of the generated samples in Figure 14 show that IWGAN seems to produce better samples than LSGAN.
Revised: Visual inspection of the generated samples in Figure 14 show that IWGAN seems to produce better samples than LSGAN.

Original: Quantitatively, we use the MNIST training set as a reference and compare the distribution of pixel intensities.
Revised: Quantitatively, we use the MNIST training set as a reference and compare the distribution of pixel intensities.

Original: Table 1 reveals that although samples generated with LSGAN and
Revised: Table 1 reveals that although samples generated with LSGAN and

Original: IWGAN look similar to the training set, they are considerably different from the training set given the
Revised: IWGAN look similar to the training set, they are considerably different from the training set given the

Original: Kolgomorov-Smirnov (KS)
Revised: Kolgomorov-Smirnov (KS)

Original: Two Sample Test and the Jensen-Shannon Divergence (JSD), specially with respect to the same statistics on the MNIST test data.
Revised: Two Sample Test and the Jensen-Shannon Divergence (JSD), specially with respect to the same statistics on the MNIST test data.

Original: [Table]
Revised: [Table]

Original: samples using

Original: MNIST training set as reference.

Original: These numerical phenomena can be understood by investigating the empirical CDFs in Figure 1.
Revised: These numerical phenomena can be understood by investigating the empirical CDFs in Figure 1.

Original: The distribution of pixel values of the samples generated with the GAN framework is mainly bi-modal and asymptotically approaches the modes of the distribution in the real data, values 0 and 1 .
Revised: The distribution of pixel values of the samples generated with the GAN framework is mainly bi-modal and asymptotically approaches the modes of the distribution in the real data, values 0 and 1 .

Original: Expectedly, the FGSM method, noted as mnist_adversarial , causes a shift on the modes of the distribution that can be easily identiﬁed.
Revised: Expectedly, the FGSM method, noted as mnist_adversarial , causes a shift on the modes of the distribution that can be easily identiﬁed.

Original: In addition, plots of the distribution of statistical moments of the spectral centroid in 2 suggests that the fake images are more noisy than the real images.
Revised: In addition, plots of the distribution of statistical moments of the spectral centroid in 2 suggests that the fake images are more noisy than the real images.

Original: Consider for example images produced by randomly sampling a Bernoulli distribution with parameter estimated from the training data.
Revised: Consider for example images produced by randomly sampling a Bernoulli distribution with parameter estimated from the training data.

Original: These images have pixel values of 0 or 1 that are equally distributed 3 over the image.
Revised: These images have pixel values of 0 or 1 that are equally distributed 3 over the image.

Original: Well, an image that has pixels values distributed in such a manner will have a distribution of mean spectral centroid with a mode at the center row of the image.
Revised: Well, an image that has pixels values distributed in such a manner will have a distribution of mean spectral centroid with a mode at the center row of the image.

Original: This and the similarity between the distribution of mean 

Revised: This and the similarity between the distribution of mean

Original: This spatial distribution is  of the  of the Bernoulli distribution.
Revised: This spatial distribution is independent of the parameter of the Bernoulli distribution.

Original: spectral centroids from fake data and Bernoulli data suggest that the fake images have noise that are also equally spatially distributed.
Revised: spectral centroids from fake data and Bernoulli data suggest that the fake images have noise that are also equally spatially distributed.

Original: Last, Figure 3 shows that the GAN generated samples smoothly approximate the modes of the distribution.
Revised: Last, Figure 3 shows that the GAN generated samples smoothly approximate the modes of the distribution.

Original: This smooth approximation is considerably different from the training and test sets.
Revised: This smooth approximation is considerably different from the training and test sets.

Original: Although not perceptually meaningful, these properties can be used to identify the source of the data.
Revised: Although not perceptually meaningful, these properties can be used to identify the source of the data.

Original: Our ﬁrst hypothesis for the smooth approximation of the modes of the distribution was that it would be present in any data produced with a generator that is trained using stochastic gradient descent and an asymptotically converging activation function, such as sigmoid or tanh, at the output of the generator.
Revised: Our ﬁrst hypothesis for the smooth approximation of the modes of the distribution was that it would be present in any data produced with a generator that is trained using stochastic gradient descent and a saturating activation function, such as sigmoid or tanh, at the output of the generator.

Original: To evaluate this hypothesis, we conducted a set of experiments using different GAN architectures (WGAN-GP, LSGAN, DCGAN) with different activation functions, including linear and the scaled tanh, at the output of the Generator, keeping the Discriminator ﬁxed.
Revised: To evaluate this hypothesis, we conducted a set of experiments using different GAN architectures (WGAN-GP, LSGAN, DCGAN) with different activation functions, including linear and the scaled tanh, at the output of the Generator, keeping the Discriminator ﬁxed.

Original: To our surprise, we noticed that the models trained with linear or scaled tanh activations were partially able to produce images that were similar to the MNIST training data and the distribution of pixel intensities, although uni-modal around zero, still possessed a smooth looking curve.
Revised: To our surprise, we noticed that the models trained with linear or scaled tanh activations were partially able to produce images that were similar to the MNIST training data and the distribution of pixel intensities, although uni-modal around zero, still possessed a smooth looking curve.

Original: This is illustrated in Figure 4.
Revised: This is illustrated in Figure 4.

Original: We then postulated that the smooth behavior was due to smoothness in the pixels intensities of the training data itself.
Revised: We then postulated that the smooth behavior was due to smoothness in the pixels intensities of the training data itself.

Original: To validate this, we binarized the real data by ﬁrst scaling it between [0 , 1] and then thresholding it at 0 .
Revised: To evaluate this, we binarized the real data by ﬁrst scaling it between [0 , 1] and then thresholding it at 0 .

Original: With this alteration the distribution of the pixel intensities of the real data becomes completely bi-modal with modes at 0 and 1 .
Revised: With this alteration the distribution of the pixel intensities of the real data becomes completely bi-modal with modes at 0 and 1 .

Original: Figure 5 shows that, as expected, the smooth behavior remained.
Revised: Figure 5 shows that the smooth behavior remained.

Original: With this empirical evidence at hand, we provide an informal analysis of the smoothness of the distribution of pixels of the generated data from the perspective of optimization, differentiation and function approximation with neural networks.
Revised: With this empirical evidence at hand, we provide an informal analysis of the smoothness of the distribution of pixels of the generated data from the perspective of optimization, differentiation and function approximation with neural networks.

Original: We know that backpropagation and stochastic gradient descent are used to update the weights of a neural network model based on the gradient of the loss with respect to the weights.
Revised: We know that backpropagation and stochastic gradient descent are used to update the weights of a neural network model based on the gradient of the loss with respect to the weights.

Original: We also know that differentiation requires the function that is being
Revised: We also know that differentiation requires the function that is being

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: differentiated to have some high degree of smoothness and be differentiable almost everywhere.
Revised: differentiated to have some high degree of smoothness and be differentiable almost everywhere.

Original: Hence, we conjecture that the inductive bias of this learning setup is that of smoothness given the requirements of differentiation.
Revised: Hence, we conjecture that the inductive bias of this learning setup is that of smoothness given the requirements of differentiation.

Original: We also speculate that this inductive bias is responsible for the smoothness of the distribution of pixel values at any iteration during training and that the U shape of the distribution of pixel values, similar to blurring, is the byproduct of an smooth approximation of the function that is being learned 4 .
Revised: We also speculate that this inductive bias is responsible for the smoothness of the distribution of pixel values at any iteration during training and that the U shape of the distribution of pixel values, similar to blurring, is the byproduct of an smooth approximation of the function that is being learned 4 .

Original: 4. CIFAR 
Revised: 4. CIFAR

Original: Expecting that the results we obtained during our MNIST experiments would generalize to other images, we brieﬂy investigate the properties of CIFAR10 fake samples generated with the IWGAN framework and using the DCGAN architecture.
Revised: Expecting that the results we obtained during our MNIST experiments would generalize to other images, we brieﬂy investigate the properties of CIFAR10 fake samples generated with the IWGAN framework and using the DCGAN architecture.

Original: The models were trained using the CIFAR10’s training set with 50k samples and following the experimental setup provided by the IWGAN authors.
Revised: The models were trained using the CIFAR10’s training set with 50k samples and following the experimental setup provided by the IWGAN authors.

Original: We report results on CIFAR10 train, test and IWGAN generated samples, all with ten thousand items each.
Revised: We report results on CIFAR10 train, test and IWGAN generated samples, all with ten thousand items each.

Original: An informal analysis of Figure 6a shows that the distribution of pixels per channel is different between the real and fake data, specially for pixels values close to − 1 .
Revised: An informal analysis of Figure 6a shows that the distribution of pixels per channel is different between the real and fake data, specially for pixels values close to − 1 .

Original: Numerically, we can see in Table 6b that the JSD between the samples from the training data and IWGAN samples is considerably large with respect to the same statistics on the test data.
Revised: Numerically, we can see in Table 6b that the JSD between the samples from the training data and IWGAN samples is considerably large with respect to the same statistics on the test data.

Original: Figure 7 shows a behavior seen in our MNIST experiments: the GAN generated samples smoothly approximate the mode of the pixel value distribution at 1 and this smooth approximation is considerably different from the training set.
Revised: Figure 7 shows a behavior seen in our MNIST experiments: the GAN generated samples smoothly approximate the mode of the pixel value distribution at 1 and this smooth approximation is considerably different from the training set.

Original: As we previously explained, these properties can be used to identify the gan samples although they might be not perceptible with the naked eye.
Revised: As we previously explained, these properties can be used to identify the GAN samples although they might be not perceptible with the naked eye.

Original: 4. B ACH C HORALES
Revised: 4. B ACH C HORALES

Original: We investigate the properties of Bach chorales generated with the GAN framework and verify if they satisfy musical speciﬁcations learned from real data.
Revised: We investigate the properties of Bach chorales generated with the GAN framework and verify if they satisfy musical speciﬁcations learned from real data.

Original: Bach chorales are polyphonic pieces of music,
Revised: Bach chorales are polyphonic pieces of music,

Original: [Figure]
Revised: [Figure]

Original: normally written for 4 or 5 voices, that follow a set of speciﬁcations or rules 5 .
Revised: normally written for 4 or 5 voices, that follow a set of speciﬁcations or rules 5 .

Original: For example, a global speciﬁcation could assert that only a set of durations are valid; a local speciﬁcation could assert that only certain transitions between states (notes) are valid depending on the current harmony.
Revised: For example, a global speciﬁcation could assert that only a set of durations are valid; a local speciﬁcation could assert that only certain transitions between states (notes) are valid depending on the current harmony.

Original: For this experiment, we convert the dataset of Bach chorales to piano rolls.
Revised: For this experiment, we convert the dataset of Bach chorales to piano rolls.

Original: The piano roll is a representation in which the rows represent note numbers, the columns represent time steps and the cell values represent note intensities.
Revised: The piano roll is a representation in which the rows represent note numbers, the columns represent time steps and the cell values represent note intensities.

Original: We compare the distribution of features computed over the training set, test set, GAN generated samples and a random baseline sampled from a Bernoulli distribution with probability equal to the normalized mean value of intensities in the training data.
Revised: We compare the distribution of features computed over the training set, test set, GAN generated samples and a random baseline sampled from a Bernoulli distribution with probability equal to the normalized mean value of intensities in the training data.

Original: After scaling and thresholding, the intensities in the training and test data are strictly bi-modal and equal to 0 or 1 .
Revised: After scaling and thresholding, the intensities in the training and test data are strictly bi-modal and equal to 0 or 1 .

Original: Figure 15 below shows training, test, IWGAN and Bernoulli samples, with modes on 0 and 1.
Revised: Figure 15 below shows training, test, IWGAN and Bernoulli samples, with modes on 0 and 1.

Original: Each dataset has approximately 1000 image patches.
Revised: Each dataset has approximately 1000 image patches.

Original: Figure 8 shows a behavior that is similar to our previous MNIST experiments: the IWGAN asymptotically approximates the modes of the distribution of intensity values.
Revised: Figure 8 shows a behavior that is similar to our previous MNIST experiments: the IWGAN asymptotically approximates the modes of the distribution of intensity values.

Original: Following, we investigate if the generated samples violate the speciﬁcations of Bach chorales.
Revised: Following, we investigate if the generated samples violate the speciﬁcations of Bach chorales.

Original: We use these piano rolls to compute boolean Chroma Peeters (2004) features and to compute an empirical
Revised: We use these piano rolls to compute boolean Chroma (Peeters, 2004) features and to compute an empirical

Original: Chroma transition matrix, where the positive entries represent existing and valid transitions.
Revised: Chroma transition matrix, where the positive entries represent existing and valid transitions.

Original: The transition matrix built on the training data is taken as the reference speciﬁcation, i.e. anything that is not included is a violation of the speciﬁcation.
Revised: The transition matrix built on the training data is taken as the reference speciﬁcation, i.e. anything that is not included is a violation of the speciﬁcation.

Original: Table 2 shows the number of violations given each dataset.
Revised: Table 2 shows the number of violations given each dataset.

Original: Although Figure 15 shows generated samples that look similar to the real data, the IWGAN samples have over 5000 violations, 10 times more than the test set!
Revised: Although Figure 15 shows generated samples that look similar to the real data, the IWGAN samples have over 5000 violations, 10 times more than the test set!

Original: Violation of speciﬁcations is a strong evidence that fake samples do not come from the same distribution as  real data.
Revised: Violation of speciﬁcations is a strong evidence that fake samples do not come from the same distribution as the real data.

Original: Furthermore, to the trained ear the fake samples violate the style of Bach.
Revised: Furthermore, to the trained ear the fake samples violate the style of Bach.

Original: We invite  readers to listen to them.
Revised: We invite the readers to listen to them.

Revised: [Figure]

Original:  In addition to experiments with Chroma features, we computed the distribution of note durations on the boolean piano roll described above.
Revised: [Table] [Figure] [Table] In addition to experiments with Chroma features, we computed the distribution of note durations on the boolean piano roll described above.

Original: Figure 9a shows the distribution of note durations within each dataset.
Revised: Figure 9a shows the distribution of note durations within each dataset.

Original: The train and test data are approximately bi-modal and, again, the improved WGAN[Figure] smoothly approximates the dominating modes of the distribution.
Revised: The train and test data are approximately bi-modal and, again, the improved WGAN smoothly approximates the dominating modes of the distribution.

Original: Table 9b provides a numerical comparison between datasets.
Revised: Table 9b provides a numerical comparison between datasets.

Original: [Figure]
Revised: [Figure]

Original: 4. S PEECH
Revised: 4. S PEECH

Original: Within the speech domain, we investigateMel-Spectrogram real and fake samples.
Revised: Within the speech domain, we investigate real and fake samples from Mel-Spectrograms.

Original: We divide the NIST 2004 dataset into training and test set, generate samples with the GAN framework and use a random baseline sampled from a Exponential distribution with parameters chosen using heuristics.
Revised: We divide the NIST 2004 dataset into training and test set, generate samples with the GAN framework and use a random baseline sampled from a Exponential distribution with parameters chosen using heuristics.

Original: The generated samples can be seen in Figure 16.
Revised: The generated samples can be seen in Figure 16.

Original: We obtain the Mel-Spectrogram by projecting a spectrogram onto a mel scale, which we do with the python library librosa McFee et al.
Revised: We obtain the Mel-Spectrogram by projecting a spectrogram onto a mel scale, which we do with the python library librosa (McFee et al., 2015).

Original: More speciﬁcally, we project the spectrogram onto 64 mel bands, with window size equal tosamples and hop size equal to 160 samples, i.e. frames of 100ms long.
Revised: More speciﬁcally, we project the spectrogram onto 64 mel bands, with window size equal tosamples and hop size equal to 160 samples, i.e. frames are 100ms long.

Original: Dynamic range compression is computed as described in Lukic et al. (2016), with log (1 + C ∗ M ) , where C is the compression constant scalar set to 1000 and M is the matrix representing the Mel-Spectrogram.
Revised: Dynamic range compression is computed as described in (Lukic et al., 2016), with log (1 + C ∗ M ) , where C is the compression constant scalar set to 1000 and M is the matrix representing the Mel-Spectrogram.

Original: Each dataset has approximately 1000 image patches and the GAN models are trained using DCGAN with the improved Wasserstein GAN algorithm.
Revised: Each dataset has approximately 1000 image patches and the GAN models are trained using DCGAN with the improved Wasserstein GAN algorithm.

Original: Figure 10a shows the empirical CDFs of intensity values.
Revised: Figure 10a shows the empirical CDFs of intensity values.

Original: Unlike our previous experiments where intensities (Bach Chorales) or pixel values (MNIST, CIFAR10) were linear and discrete, in this experiment intensities are continuous and compressed using the log function.
Revised: Unlike our previous experiments where intensities (Bach Chorales) or pixel values (MNIST, CIFAR10) were linear and discrete, in this experiment intensities are continuous and compressed using the log function.

Original: This considerably reduces the distance between the empirical CDFs of the training data and GAN samples, specially around the saturating points of the tanh non-linearity, − 1 and 1 in this case. 
Revised: This considerably reduces the distance between the empirical CDFs of the training data and GAN samples, specially around the saturating points of the tanh non-linearity, − 1 and 1 in this case. [Figure]

Original:  Table 10b shows a signiﬁcant difference between the KS-Statistic of test samples and fake samples with respect to the training data.
Revised: [Figure] Table 10b shows a signiﬁcant difference between the KS-Statistic of test samples and fake samples with respect to the training data.

Original: However, an adversary can manipulate the fake samples to considerably decrease this difference and still keep the high similarity in features harder to simulate such as moments of spectral centroid or slope.[Figure] [Figure]
Revised: However, an adversary can manipulate the fake samples to considerably decrease this difference and still keep the high similarity in features harder to simulate such as moments of spectral centroid or slope.

Original: Figure 11 shows the distribution of statistical moments computed on spectral centroids and slope.
Revised: Figure 11 shows the distribution of statistical moments computed on spectral centroids and slope.

Original: The distributions from different sources considerably overlap, indicating that the generator has efﬁciently approximated the real distribution of these features.
Revised: The distributions from different sources considerably overlap, indicating that the generator has efﬁciently approximated the real distribution of these features.

Original: [Figure]
Revised: [Figure]

Original:  C ONCLUSIONS
Revised:  C ONCLUSIONS

Original: In this paper we investigated numerical properties of samples produced with adversarial methods, specially Generative Adversarial Networks.
Revised: In this paper we investigated numerical properties of samples produced with adversarial methods, specially Generative Adversarial Networks.

Original: We showed that fake samples have properties that are barely noticed with visual  of samples, namely the fact that, due to stochastic gradient descent and the requirements of differentiability, fake samples smoothly approximate the dominating modes of the distribution.
Revised: We showed that fake samples have properties that are barely noticed with visual inspection of samples, namely the fact that fake samples smoothly approximate the dominating modes of the distribution due to stochastic gradient descent and the requirements of differentiability.

Original: We analysed statistical measures of divergence between real data and other data and the results showed that even in simple cases, e.g. distribution of pixel intensities, the divergence between training data and fake data is large with respect to test data.
Revised: We analysed statistical measures of divergence between real data and other data and the results showed that even in simple cases, e.g. distribution of pixel intensities, the divergence between training data and fake data is large with respect to test data.

Original: Finally, we mined speciﬁcations from real data and showed that, unlike test data, the fake data considerably violates the speciﬁcations of the real data.
Revised: Finally, we mined speciﬁcations from real data and showed that, unlike test data, the fake data considerably violates the speciﬁcations of the real data.

Original: In the context of adversarial attacks, these large differences in distribution and specially violations of speciﬁcation can be used to identify data that is fake.
Revised: In the context of adversarial attacks, these large differences in distribution and specially violations of speciﬁcation can be used to identify data that is fake.

Original: In our results we show that, although some of the features used to learn speciﬁcations in this paper are weakly perceptually correlated with the content of the image, they certainly can be used to identify fake samples.
Revised: In our results we show that, although some of the features used to learn speciﬁcations in this paper are weakly perceptually correlated with the content of the image, they certainly can be used to identify fake samples.

Original: Although not common practice, one could possibly circumvent the difference in support between the real and fake data by training Generators that explicitly sample a distribution that replicates the support of the real data, i.e. 256 values in the case of discretized images.
Revised: Although not common practice, one could possibly circumvent the difference in support between the real and fake data by training Generators that explicitly sample a distribution that replicates the support of the real data, i.e. 256 values in the case of discretized images.

Original: Conversely, one could mine speciﬁcations that are easy to learn from real data but hardly differentiable.
Revised: Conversely, one could mine speciﬁcations that are easy to learn from real data but hardly differentiable.

Original: These are topics that are not limited to GANs and remain to be explored in the larger domain of Veriﬁed Artiﬁcial Intelligence considered in Seshia & Sadigh (2016).
Revised: These are topics that are not limited to GANs and remain to be explored in the larger domain of Veriﬁed Artiﬁcial Intelligence considered in (Seshia & Sadigh, 2016).

Original: A CKNOWLEDGMENTS
Revised: A CKNOWLEDGMENTS

Original: We are thankful to Ryan Prenger and Kevin Shih for their feedback on this paper.
Revised: We are thankful to Ryan Prenger and Kevin Shih for their feedback on this paper.

Original: We acknowledge
Revised: We acknowledge

Original: NVIDIA for providing us with the Titan X GPU used in these experiments.
Revised: NVIDIA for providing us with the Titan X GPU used in these experiments.

Original: A PPENDIX A S PECTRAL C ENTROID AND S LOPE I MAGES
Revised: A PPENDIX A S PECTRAL C ENTROID AND S LOPE I MAGES

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: A PPENDIX

Original: [Figure]
Revised: [Figure]

Original: MNIST I MAGES

Original: [Figure] [Figure]
Revised: [Figure] [Figure]