Original: Q UANTITATIVELY E VALUATING GAN S W ITH D IVERGENCES P ROPOSED FOR T RAINING
Revised: Q UANTITATIVELY E VALUATING GAN S W ITH D IVERGENCES P ROPOSED FOR T RAINING

Original: A BSTRACT
Revised: A BSTRACT

Original: Generative adversarial networks (GANs) have been extremely effective in approximating complex distributions of high-dimensional, input data samples, and substantial progress has been made in understanding and improving GAN performance in terms of both theory and application.
Revised: Generative adversarial networks (GANs) have been extremely effective in approximating complex distributions of high-dimensional, input data samples, and substantial progress has been made in understanding and improving GAN performance in terms of both theory and application.

Original: However, we currently lack quantitative methods for model assessment.
Revised: However, we currently lack quantitative methods for model assessment.

Original: Because ot this, while many GAN variants being proposed, we have relatively little understanding of their relative abilities.
Revised: Because ot this, while many GAN variants being proposed, we have relatively little understanding of their relative abilities.

Original: In this paper, we evaluate the performance of various types of GANs using divergence and distance functions typically used only for training.
Revised: In this paper, we evaluate the performance of various types of GANs using divergence and distance functions typically used only for training.

Original: We observe consistency across the various proposed metrics and, interestingly, the test-time metrics do not favour networks that use the same training-time criterion.
Revised: We observe consistency across the various proposed metrics and, interestingly, the test-time metrics do not favour networks that use the same training-time criterion.

Original: We also compare the proposed metrics to human perceptual scores.
Revised: We also compare the proposed metrics to human perceptual scores.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Generative adversarial networks (GANs) aim to approximate a data distribution P , using a parameterized model distribution Q .
Revised: Generative adversarial networks (GANs) aim to approximate a data distribution P , using a parameterized model distribution Q .

Original: They achieve this by jointly optimizing generative and discriminative networks (Goodfellow et al., 2014).
Revised: They achieve this by jointly optimizing generative and discriminative networks (Goodfellow et al., 2014).

Original: GANs are end-to-end differentiable, and samples from the generative network are propagated forward to a discriminative network, and error signals are then propagated backwards from the discriminative network to the generative network.
Revised: GANs are end-to-end differentiable, and samples from the generative network are propagated forward to a discriminative network, and error signals are then propagated backwards from the discriminative network to the generative network.

Original: The discriminative network is often viewed as learned, adaptive loss function for the generative network.
Revised: The discriminative network is often viewed as learned, adaptive loss function for the generative network.

Original: GANs have achieved state-of-the-art results for a number of applications (Goodfellow, 2016), producing more realistic, sharper samples than other popular generative models, such as variational autoencoders (Kingma & Welling, 2014).
Revised: GANs have achieved state-of-the-art results for a number of applications (Goodfellow, 2016), producing more realistic, sharper samples than other popular generative models, such as variational autoencoders (Kingma & Welling, 2014).

Original: Because of their success, many GAN frameworks have been proposed.
Revised: Because of their success, many GAN frameworks have been proposed.

Original: However, it has been difﬁcult to compare these algorithms and understand their strengths and weaknesses because we are currently lacking in quantitative methods for assessing the learned generators.
Revised: However, it has been difﬁcult to compare these algorithms and understand their strengths and weaknesses because we are currently lacking in quantitative methods for assessing the learned generators.

Original: In this work, we propose new metrics for measuring how realistic samples generated from GANs are.
Revised: In this work, we propose new metrics for measuring how realistic samples generated from GANs are.

Original: These criteria are based on a formulation of divergence between the distributions P and Q (Nowozin et al., 2016; Sriperumbudur et al., 2009):
Revised: These criteria are based on a formulation of divergence between the distributions P and Q (Nowozin et al., 2016; Sriperumbudur et al., 2009):

Original: [Equation]
Revised: [Equation]

Original: Here, different choices of µ , υ , and F can correspond to different f -divergences
Revised: Here, different choices of µ , υ , and F can correspond to different f -divergences

Original: (Nowozin et al., 2016) or different integral probability metrics (IPMs) (Sriperumbudur et al., 2009).
Revised: (Nowozin et al., 2016) or different integral probability metrics (IPMs) (Sriperumbudur et al., 2009).

Original: Importantly,
Revised: Importantly,

Original: J ( Q ) can be estimated using samples from P and Q , and does not require us to be able to estimate
Revised: J ( Q ) can be estimated using samples from P and Q , and does not require us to be able to estimate

Original: P ( x ) or Q ( x ) for samples x .
Revised: P ( x ) or Q ( x ) for samples x .

Original: Instead, evaluating J ( Q ) involves ﬁnding the function f ∈ F that is maximally different with respect to P and Q .
Revised: Instead, evaluating J ( Q ) involves ﬁnding the function f ∈ F that is maximally different with respect to P and Q .

Original: This measure of divergence between the distributions P and Q is related to the GAN criterion if we restrict the function class F to be neural network functions parameterized by the vector φ and the class of approximating distributions to correspond to neural network generators G θ parameterized by the vector θ , allowing formulation as a min-max problem:
Revised: This measure of divergence between the distributions P and Q is related to the GAN criterion if we restrict the function class F to be neural network functions parameterized by the vector φ and the class of approximating distributions to correspond to neural network generators G θ parameterized by the vector θ , allowing formulation as a min-max problem:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: [Table]
Revised: [Table]

Original: In this formulation, Q θ corresponds to the generator network’s distribution and D φ corresponds to the discriminator network (see (Nowozin et al., 2016) for details).
Revised: In this formulation, Q θ corresponds to the generator network’s distribution and D φ corresponds to the discriminator network (see (Nowozin et al., 2016) for details).

Original: We propose using J ( θ ) to evaluate the performance of the generator network G θ for various choices of µ and υ , corresponding to different f -divergences or IPMs between distributions P and Q θ , that have been successfully used for GAN training.
Revised: We propose using J ( θ ) to evaluate the performance of the generator network G θ for various choices of µ and υ , corresponding to different f -divergences or IPMs between distributions P and Q θ , that have been successfully used for GAN training.

Original: Our proposed metrics differ from most existing metrics in that they are adaptive, and involve ﬁnding the maximum over discriminative networks.
Revised: Our proposed metrics differ from most existing metrics in that they are adaptive, and involve ﬁnding the maximum over discriminative networks.

Original: We compare four metrics, those corresponding to the original GAN (GC) (Goodfellow, 2016), the
Revised: We compare four metrics, those corresponding to the original GAN (GC) (Goodfellow, 2016), the

Original: Least-Squares GAN (LS) (Mao et al., 2017),the Wasserstein GAN (IW) (Arjovsky et al., 2017), and the Maximum Mean Discrepency GAN (MMD) (Li et al., 2017) criteria.
Revised: Least-Squares GAN (LS) (Mao et al., 2017),the Wasserstein GAN (IW) (Arjovsky et al., 2017), and the Maximum Mean Discrepency GAN (MMD) (Li et al., 2017) criteria.

Original: Choices for µ , υ , and F for these metrics are shown in Table 1.
Revised: Choices for µ , υ , and F for these metrics are shown in Table 1.

Original: Our method can easily be extended to other f -divergences or
Revised: Our method can easily be extended to other f -divergences or

Original: IPMs.
Revised: IPMs.

Original: To compare these and previous metrics for evaluating GANs, we performed many experiments, training and comparing multiple types of GANs with multiple architectures on multiple data sets.
Revised: To compare these and previous metrics for evaluating GANs, we performed many experiments, training and comparing multiple types of GANs with multiple architectures on multiple data sets.

Original: We qualitatively and quantitatively compared these metrics to human perception, and found that our proposed metrics better reﬂected human perception.
Revised: We qualitatively and quantitatively compared these metrics to human perception, and found that our proposed metrics better reﬂected human perception.

Original: We also show that rankings produced using our proposed metrics are consistent across metrics, thus are robust to the exact choices of the functions µ and υ in Equation 2.
Revised: We also show that rankings produced using our proposed metrics are consistent across metrics, thus are robust to the exact choices of the functions µ and υ in Equation 2.

Original: We used the proposed metrics to quantitatively analyze three different families of GANs: Deep
Revised: We used the proposed metrics to quantitatively analyze three different families of GANs: Deep

Original: Convolutional Generative Adversarial Networks (DCGAN) (Radford et al., 2015), Least-Squares
Revised: Convolutional Generative Adversarial Networks (DCGAN) (Radford et al., 2015), Least-Squares

Original: GANs (LS-DCGAN), and Wasserstein GANs (W-DCGAN), each of which corresponded to a different proposed metric.
Revised: GANs (LS-DCGAN), and Wasserstein GANs (W-DCGAN), each of which corresponded to a different proposed metric.

Original: Interestingly, we found that the different proposed metrics still agreed on the best
Revised: Interestingly, we found that the different proposed metrics still agreed on the best

Original: GAN framework for each dataset.
Revised: GAN framework for each dataset.

Original: Thus, even though, e.g. for MNIST the W-DCGAN was trained with the IW criterion, LS-DCGAN still outperformed it for the IW criterion.
Revised: Thus, even though, e.g. for MNIST the W-DCGAN was trained with the IW criterion, LS-DCGAN still outperformed it for the IW criterion.

Original: Our analysis also included carrying out a sensitivity analysis with respect to various factors, such as the architecture size, noise dimension, update ratio between discriminator and generator, and number of data points.
Revised: Our analysis also included carrying out a sensitivity analysis with respect to various factors, such as the architecture size, noise dimension, update ratio between discriminator and generator, and number of data points.

Original: Our empirical results show that: i) the larger the GAN architecture, the better the results; ii) having a generator network larger than the discriminator network does not yield good results; iii) the best ratio between discriminator and generator updates depend on the data set; and iv) the W-DCGAN and LS-DCGAN performance increases much faster than DCGAN as the number of training examples grows.
Revised: Our empirical results show that: i) the larger the GAN architecture, the better the results; ii) having a generator network larger than the discriminator network does not yield good results; iii) the best ratio between discriminator and generator updates depend on the data set; and iv) the W-DCGAN and LS-DCGAN performance increases much faster than DCGAN as the number of training examples grows.

Original: These metrics thus allow us to tune the hyper-parameters and architectures of GANs based on our proposed method.
Revised: These metrics thus allow us to tune the hyper-parameters and architectures of GANs based on our proposed method.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: GANs can be evaluated using manual annotations, but this is time consuming and difﬁcult to reproduce.
Revised: GANs can be evaluated using manual annotations, but this is time consuming and difﬁcult to reproduce.

Original: Several automatically computable metrics have been proposed for evaluating the performance of probabilistic general models and GANs in particular.
Revised: Several automatically computable metrics have been proposed for evaluating the performance of probabilistic general models and GANs in particular.

Original: We review some of these here, and compare our proposed metrics to these in our experiments.
Revised: We review some of these here, and compare our proposed metrics to these in our experiments.

Original: Many previous probabilistic generative models were evaluated based on the pointwise likelihood of the test data, the criterion also used during training.
Revised: Many previous probabilistic generative models were evaluated based on the pointwise likelihood of the test data, the criterion also used during training.

Original: While GANs can be used to generate samples from the approximate distribution, its likelihood on test samples cannot be evaluated without simplifying assumptions.
Revised: While GANs can be used to generate samples from the approximate distribution, its likelihood on test samples cannot be evaluated without simplifying assumptions.

Original: As discussed in (Theis et al., 2015), likelihood often does not provide good rankings of how realistic samples look, the main goal of GANs.
Revised: As discussed in (Theis et al., 2015), likelihood often does not provide good rankings of how realistic samples look, the main goal of GANs.

Original: We evaluted the efﬁcacy of the log-likelihood of the test data, as estimated using Annealed Importance Sampling (AIS)
Revised: We evaluted the efﬁcacy of the log-likelihood of the test data, as estimated using Annealed Importance Sampling (AIS)

Original: (Wu et al., 2016).
Revised: (Wu et al., 2016).

Original: AIS has been to estimate the likelihood of a test sample x by considering many intermediate distributions that are deﬁned by taking a weighted geometric mean between the prior (input) distribution, p ( z ) , and an approximation of the joint distribution p σ ( x, z ) =
Revised: AIS has been to estimate the likelihood of a test sample x by considering many intermediate distributions that are deﬁned by taking a weighted geometric mean between the prior (input) distribution, p ( z ) , and an approximation of the joint distribution p σ ( x, z ) =

Original: p σ ( x | z ) p ( z ) .
Revised: p σ ( x | z ) p ( z ) .

Original: Here, p σ ( x | z ) is a Gaussian kernel with ﬁxed standard deviation σ around mean G θ ( z ) .
Revised: Here, p σ ( x | z ) is a Gaussian kernel with ﬁxed standard deviation σ around mean G θ ( z ) .

Original: The ﬁnal estimate depends critically on the accuracy of this approximation.
Revised: The ﬁnal estimate depends critically on the accuracy of this approximation.

Original: In Section 4, we demonstrate that the AIS estimate of p ( x ) is highly dependent on the choice of this hyperparameter.
Revised: In Section 4, we demonstrate that the AIS estimate of p ( x ) is highly dependent on the choice of this hyperparameter.

Original: The Generative Adversarial Metric (Im et al., 2016a) measures the relative performance of two
Revised: The Generative Adversarial Metric (Im et al., 2016a) measures the relative performance of two

Original: GANs by measuring the likelihood ratio of the two models.
Revised: GANs by measuring the likelihood ratio of the two models.

Original: Consider two GANs with their respective trained partners, M 1 = ( D 1 , G 1 ) and M 2 = ( D 2 , G 2 ) , where G 1 and G 2 are the generators and Dand D 2 are the discriminators.
Revised: Consider two GANs with their respective trained partners, M 1 = ( D 1 , G 1 ) and M 2 = ( D 2 , G 2 ) , where G 1 and G 2 are the generators and Dand D 2 are the discriminators.

Original: The hypothesis H 1 is that M 1 is better than M 2 if G 1 fools D 2 more than G 2 fools D 1 , and vice versa for the hypothesis H 0 .
Revised: The hypothesis H 1 is that M 1 is better than M 2 if G 1 fools D 2 more than G 2 fools D 1 , and vice versa for the hypothesis H 0 .

Original: The likelihood-ratio is deﬁned as:
Revised: The likelihood-ratio is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: where M (cid:48) 1 and M (cid:48) 2 are the swapped pairs ( D 1 , G 2 ) and ( D 2 , G 1 ) , and p ( x | y
Revised: where M (cid:48) 1 and M (cid:48) 2 are the swapped pairs ( D 1 , G 2 ) and ( D 2 , G 1 ) , and p ( x | y

Original: = 1 , M ) is the likelihood of x generated from the data distribution p ( x ) by model M and p ( y = 1 | x ; D ) indicates that discriminator D thinks x is a real sample.
Revised: = 1 , M ) is the likelihood of x generated from the data distribution p ( x ) by model M and p ( y = 1 | x ; D ) indicates that discriminator D thinks x is a real sample.

Original: To evaluate this, we measure the ratio of how frequently
Revised: To evaluate this, we measure the ratio of how frequently

Original: G 1 , the generator from model 1, fools D 2 , the discriminator from model 2, and vice-versa: D 1 ( x 2 ) D 2 ( x 1 ) , where x 1 ∼ G 1 and x 2 ∼ G 2 .
Revised: G 1 , the generator from model 1, fools D 2 , the discriminator from model 2, and vice-versa: D 1 ( x 2 ) D 2 ( x 1 ) , where x 1 ∼ G 1 and x 2 ∼ G 2 .

Original: There are two main caveats to the Generative Adversarial Metric.
Revised: There are two main caveats to the Generative Adversarial Metric.

Original: First, the measurement only provides comparisons between pairs of models.
Revised: First, the measurement only provides comparisons between pairs of models.

Original: Second, the metric has a constraint where the two discriminators must have an approximately similar performance on a calibration dataset, which can be difﬁcult to satisfy in practice.
Revised: Second, the metric has a constraint where the two discriminators must have an approximately similar performance on a calibration dataset, which can be difﬁcult to satisfy in practice.

Original: The Inception Score (Salimans et al., 2016) (IS) measures the performance of a model using a third-party neural network trained on a supervised classiﬁcation task, e.g. Imagenet.
Revised: The Inception Score (Salimans et al., 2016) (IS) measures the performance of a model using a third-party neural network trained on a supervised classiﬁcation task, e.g. Imagenet.

Original: The IS computes the expectation of divergence between the distribution of class predictions for samples from the GAN compared to the distribution of class to the distribution of class labels used to train the third-party network,
Revised: The IS computes the expectation of divergence between the distribution of class predictions for samples from the GAN compared to the distribution of class to the distribution of class labels used to train the third-party network,

Original: [Equation]
Revised: [Equation]

Original: Here, the class prediction given a sample x is computed using the third-party neural network.
Revised: Here, the class prediction given a sample x is computed using the third-party neural network.

Original: In (Salimans et al., 2016), Google’s Inception Network (Szegedy et al., 2015) trained on Imagenet was the third-party neural network.
Revised: In (Salimans et al., 2016), Google’s Inception Network (Szegedy et al., 2015) trained on Imagenet was the third-party neural network.

Original: IS is the most widely used metric to measure GAN performance.
Revised: IS is the most widely used metric to measure GAN performance.

Original: However, summarizing samples as the class prediction from a network trained for a different task discards much of the important information in the sample.
Revised: However, summarizing samples as the class prediction from a network trained for a different task discards much of the important information in the sample.

Original: In addition, it requires another neural network that is trained separately via supervised learning.
Revised: In addition, it requires another neural network that is trained separately via supervised learning.

Original: We demonstrate an example of a failure case of IS in the Experiments section.
Revised: We demonstrate an example of a failure case of IS in the Experiments section.

Original: The Fréchet Inception Distance (FID) (Heusel et al., 2017) extends upon IS.
Revised: The Fréchet Inception Distance (FID) (Heusel et al., 2017) extends upon IS.

Original: Instead of using the ﬁnal classiﬁcation outputs from the third-party network as representations of samples, it uses a representation computed from a late layer of the third-party network.
Revised: Instead of using the ﬁnal classiﬁcation outputs from the third-party network as representations of samples, it uses a representation computed from a late layer of the third-party network.

Original: It compares the mean m Q and covariance C Q of the Inception-based representation of samples generated by the GAN to the mean m P and covariance C P of the same representation for training samples:
Revised: It compares the mean m Q and covariance C Q of the Inception-based representation of samples generated by the GAN to the mean m P and covariance C P of the same representation for training samples:

Original: [Equation]
Revised: [Equation]

Original: This method relies on the Inception-based representation of the samples capturing all important information and the ﬁrst two moments of the distributions being descriptive of the distribution.
Revised: This method relies on the Inception-based representation of the samples capturing all important information and the ﬁrst two moments of the distributions being descriptive of the distribution.

Original: Classiﬁer Two-Sample Tests (C2ST) (Lopez-Paz & Oquab, 2016) proposes training a classiﬁer, similar to a discriminator, that can distinguish real samples from P from generated samples from
Revised: Classiﬁer Two-Sample Tests (C2ST) (Lopez-Paz & Oquab, 2016) proposes training a classiﬁer, similar to a discriminator, that can distinguish real samples from P from generated samples from

Original: Q , and using the error rate of this classiﬁer as a measure of GAN performance.
Revised: Q , and using the error rate of this classiﬁer as a measure of GAN performance.

Original: In their work, they used single-layer and k -nearest neighbor (KNN) classiﬁers trained on a representation of the samples computed from a late layer of a third-party network (in this case, ResNet (He et al., 2015)).
Revised: In their work, they used single-layer and k -nearest neighbor (KNN) classiﬁers trained on a representation of the samples computed from a late layer of a third-party network (in this case, ResNet (He et al., 2015)).

Original: C2ST is an IPM (Sriperumbudur et al., 2009), like the MMD and Wasserstein metrics we propose, with µ ( f ) =
Revised: C2ST is an IPM (Sriperumbudur et al., 2009), like the MMD and Wasserstein metrics we propose, with µ ( f ) =

Original: f and υ ( f ) = f , but with a different function class F , corresponding to the family of classiﬁers chosen (in this case, single-layer networks or KNN, see see our detailed explanation in
Revised: f and υ ( f ) = f , but with a different function class F , corresponding to the family of classiﬁers chosen (in this case, single-layer networks or KNN, see see our detailed explanation in

Original: Appendix 5).
Revised: Appendix 5).

Original: The accuracy of a classiﬁer trained to distinguish samples from distributions P and Q is just one way to measure the distance between these distributions, and, in this work, we propose a general family.
Revised: The accuracy of a classiﬁer trained to distinguish samples from distributions P and Q is just one way to measure the distance between these distributions, and, in this work, we propose a general family.

Original:  E VALUATION M ETRICS
Revised:  E VALUATION M ETRICS

Original: Given a generator G θ with parameters θ which generates samples from the distribution Q θ , we propose to measure the quality of G θ by estimating divergence between the true data distribution P and Q θ for different choices of divergence measure.
Revised: Given a generator G θ with parameters θ which generates samples from the distribution Q θ , we propose to measure the quality of G θ by estimating divergence between the true data distribution P and Q θ for different choices of divergence measure.

Original: We train both G θ and D ϕ on a training data set, and measure performance on a separate test set.
Revised: We train both G θ and D ϕ on a training data set, and measure performance on a separate test set.

Original: See Algorithm 1 for details.
Revised: See Algorithm 1 for details.

Original: We consider metrics from two widely studied divergence and distance measures, f -divergence
Revised: We consider metrics from two widely studied divergence and distance measures, f -divergence

Original: (Nguyen et al., 2008) and the Integral Probability Metric (IPM) (Muller, 1997).
Revised: (Nguyen et al., 2008) and the Integral Probability Metric (IPM) (Muller, 1997).

Original: In our experiments, we consider the following four metrics that are commonly used to train GANs.
Revised: In our experiments, we consider the following four metrics that are commonly used to train GANs.

Original: Below, ϕ represents the parameters of the discriminator network and θ represents the parameters of the generator network.
Revised: Below, ϕ represents the parameters of the discriminator network and θ represents the parameters of the generator network.

Original: Original GAN Criterion (GC)
Revised: Original GAN Criterion (GC)

Original: Training a standard GAN corresponds to minimizing the following (Goodfellow et al., 2014):
Revised: Training a standard GAN corresponds to minimizing the following (Goodfellow et al., 2014):

Original: [Equation]
Revised: [Equation]

Original: where p ( z ) is the prior distribution of the generative network and G θ ( z ) is a differentiable function from z to the data space represented by a neural network with parameter θ .
Revised: where p ( z ) is the prior distribution of the generative network and G θ ( z ) is a differentiable function from z to the data space represented by a neural network with parameter θ .

Original: D ϕ is trained with a sigmoid activation function, thus its output is guaranteed to be positive.
Revised: D ϕ is trained with a sigmoid activation function, thus its output is guaranteed to be positive.

Original: Least-Squares GAN Criterion (LS)
Revised: Least-Squares GAN Criterion (LS)

Original: A Least-Squares GAN corresponds to training with a Pearson χ 2 divergence (Mao et al., 2017):
Revised: A Least-Squares GAN corresponds to training with a Pearson χ 2 divergence (Mao et al., 2017):

Original: [Equation]
Revised: [Equation]

Original: Following (Mao et al., 2017), we set a = 0 and b = 1 when training D ϕ .
Revised: Following (Mao et al., 2017), we set a = 0 and b = 1 when training D ϕ .

Original: Maximum Mean Discrepancy (MMD)
Revised: Maximum Mean Discrepancy (MMD)

Original: The maximum mean discrepancy metric considers the largest difference in the expectations over a unit ball of RKHS H ,
Revised: The maximum mean discrepancy metric considers the largest difference in the expectations over a unit ball of RKHS H ,

Original: [Equation]
Revised: [Equation]

Original: where H is the RKHS with kernel k ( · , · ) (Gretton et al., 2012).
Revised: where H is the RKHS with kernel k ( · , · ) (Gretton et al., 2012).

Original: In this case, we do not need to train a discriminator D ϕ to evaluate our metric.
Revised: In this case, we do not need to train a discriminator D ϕ to evaluate our metric.

Original: Improved Wasserstein Distance (IW)
Revised: Improved Wasserstein Distance (IW)

Original: Arjovsky & Bottou (2017) proposed the use of the dual representation of the Wasserstein distance (Villani, 2009) for training GANs.
Revised: Arjovsky & Bottou (2017) proposed the use of the dual representation of the Wasserstein distance (Villani, 2009) for training GANs.

Original: The Wasserstein distance is an IPM which considers the 1-Lipschitz function class ϕ :
Revised: The Wasserstein distance is an IPM which considers the 1-Lipschitz function class ϕ :

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Note that IW (Danihelka et al., 2017) and MMD (Sutherland et al., 2017) were recently proposed to evaluate GANs, but have not been compared before.
Revised: Note that IW (Danihelka et al., 2017) and MMD (Sutherland et al., 2017) were recently proposed to evaluate GANs, but have not been compared before.

Original: Algorithm
Revised: Algorithm

Original: 1: procedure D IVERGENCE C OMPUTATION (Dataset { X tr , X te } , generator G θ , learning rate η , evaluation criterion J ( ϕ, X, Y ) ) 2:
Revised: 1: procedure D IVERGENCE C OMPUTATION (Dataset { X tr , X te } , generator G θ , learning rate η , evaluation criterion J ( ϕ, X, Y ) ) 2:

Original: Initialize critic network parameter ϕ .
Revised: Initialize critic network parameter ϕ .

Original: 3: for i = 1 · · · N do 4:
Revised: 3: for i = 1 · · · N do 4:

Original: Sample data points from X, { x m } ∼ X tr .
Revised: Sample data points from X, { x m } ∼ X tr .

Original: Sample points from generative model, { s m } ∼ G θ .
Revised: Sample points from generative model, { s m } ∼ G θ .

Original: 6: ϕ ← ϕ + η ∇ ϕ J ( { x m } , { s m } ; ϕ ) .
Revised: 6: ϕ ← ϕ + η ∇ ϕ J ( { x m } , { s m } ; ϕ ) .

Original: Sample points from generative model, { s m } ∼ G θ .
Revised: Sample points from generative model, { s m } ∼ G θ .

Original: 8: return J ( ϕ, X te , { s m } ) .
Revised: 8: return J ( ϕ, X te , { s m } ) .

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: The goals in our experiments are two-fold.
Revised: The goals in our experiments are two-fold.

Original: First, we wanted to evaluate the metrics we proposed for evaluating GANs.
Revised: First, we wanted to evaluate the metrics we proposed for evaluating GANs.

Original: Second, we wanted to use these metrics to evaluate GAN frameworks and architectures.
Revised: Second, we wanted to use these metrics to evaluate GAN frameworks and architectures.

Original: In particular, we evaluated how size of the discriminator and generator networks affected performance, and the sensitivity of each algorithm to training data set size.
Revised: In particular, we evaluated how size of the discriminator and generator networks affected performance, and the sensitivity of each algorithm to training data set size.

Original: GAN frameworks .
Revised: GAN frameworks .

Original: We conducted our experiments on three types of GANs: Deep Convolutional
Revised: We conducted our experiments on three types of GANs: Deep Convolutional

Original: Generative Adversarial Networks (DCGAN), Least-Squares GANs (LS-DCGAN), and Wasserstein
Revised: Generative Adversarial Networks (DCGAN), Least-Squares GANs (LS-DCGAN), and Wasserstein

Original: GANs (W-DCGAN).
Revised: GANs (W-DCGAN).

Original: Note that to not confuse the test metric names with the GAN frameworks we evaluated, we use different abbreviations.
Revised: Note that to not confuse the test metric names with the GAN frameworks we evaluated, we use different abbreviations.

Original: GC is the original GAN criterion, which is used to train DCGANs.
Revised: GC is the original GAN criterion, which is used to train DCGANs.

Original: The LS criterion is used to train the LS-DCGAN, and the IW is used to train the
Revised: The LS criterion is used to train the LS-DCGAN, and the IW is used to train the

Original: W-DCGAN.
Revised: W-DCGAN.

Original: Evaluation criteria .
Revised: Evaluation criteria .

Original: We evaluated these three families of GANs with six metrics.
Revised: We evaluated these three families of GANs with six metrics.

Original: We compared our four proposed metrics to the two most commonly used metrics for evaluating GANs, the IS and FID.
Revised: We compared our four proposed metrics to the two most commonly used metrics for evaluating GANs, the IS and FID.

Original: Because the optimization of a discriminator is required both during training and test time, we will call the discriminator learned for evaluaton of our metrics the critic , in order to not confuse the two discriminators.
Revised: Because the optimization of a discriminator is required both during training and test time, we will call the discriminator learned for evaluaton of our metrics the critic , in order to not confuse the two discriminators.

Original: We also compared these metrics to human perception, and had three volunteers evaluate and compare sets of images, either from the training data set or generated from different GAN frameworks during training.
Revised: We also compared these metrics to human perception, and had three volunteers evaluate and compare sets of images, either from the training data set or generated from different GAN frameworks during training.

Original: Data sets .
Revised: Data sets .

Original: In our experiments, we considered the MNIST (LeCun et al., 1998), CIFAR10, LSUN
Revised: In our experiments, we considered the MNIST (LeCun et al., 1998), CIFAR10, LSUN

Original: Bedroom, and Fashion MNIST datasets.
Revised: Bedroom, and Fashion MNIST datasets.

Original: MNIST consists of 60,000 training and 10,000 test images with a size of 28 × 28 pixels, containing handwritten digits from the classes 0 to 9.
Revised: MNIST consists of 60,000 training and 10,000 test images with a size of 28 × 28 pixels, containing handwritten digits from the classes 0 to 9.

Original: From the 60, training examples, we set aside 10,000 as validation examples to tune various hyper-parameters.
Revised: From the 60, training examples, we set aside 10,000 as validation examples to tune various hyper-parameters.

Original: Similarly, FashionMNIST consists exactly the same number of training and test examples.
Revised: Similarly, FashionMNIST consists exactly the same number of training and test examples.

Original: Each example is a 28x28 grayscale image, associated with a label from 10 classes.
Revised: Each example is a 28x28 grayscale image, associated with a label from 10 classes.

Original: The CIFAR10 datasetconsists of images with a size of 32 × 32 × 3 pixels, with ten different classes of objects.
Revised: The CIFAR10 datasetconsists of images with a size of 32 × 32 × 3 pixels, with ten different classes of objects.

Original: We used 45,000, 5,000, and 10,000 examples as training, validation, and test data, respectively.
Revised: We used 45,000, 5,000, and 10,000 examples as training, validation, and test data, respectively.

Original: The LSUN
Revised: The LSUN

Original: Bedroom dataset consists of images with a size of 64 × 64 pixels, depicting various bedrooms.
Revised: Bedroom dataset consists of images with a size of 64 × 64 pixels, depicting various bedrooms.

Original: From the 3,033,342 images, we used 90,000 images as training data and 90,000 images as validation data.
Revised: From the 3,033,342 images, we used 90,000 images as training data and 90,000 images as validation data.

Original: The learning rate was selected from discrete ranges and chosen based on a held-out validation set.
Revised: The learning rate was selected from discrete ranges and chosen based on a held-out validation set.

Original: Hyperparameters .
Revised: Hyperparameters .

Original: Table 10 in the Appendix shows the learning rates and the convolutional kernel sizes that were used for each experiment.
Revised: Table 10 in the Appendix shows the learning rates and the convolutional kernel sizes that were used for each experiment.

Original: The architecture of each network is presented in the
Revised: The architecture of each network is presented in the

Original: Appendix in Figure 10.
Revised: Appendix in Figure 10.

Original: Additionally, we used exponential-mean-square kernels with several different sigma values for MMD.
Revised: Additionally, we used exponential-mean-square kernels with several different sigma values for MMD.

Original: A pre-trained logistic regression and pre-trained residual network were used for IS and FID on the MNIST and CIFAR10 datasets, respectively.
Revised: A pre-trained logistic regression and pre-trained residual network were used for IS and FID on the MNIST and CIFAR10 datasets, respectively.

Original: For every experiment, we retrained 10 times with different random seeds, and report the mean and standard deviation.
Revised: For every experiment, we retrained 10 times with different random seeds, and report the mean and standard deviation.

Original: 4. Q UALITATIVE O BSERVATIONS ABOUT E XISTING M ETRICS
Revised: 4. Q UALITATIVE O BSERVATIONS ABOUT E XISTING M ETRICS

Original: The log-likelihood measurement is the most commonly used metric for generative models.
Revised: The log-likelihood measurement is the most commonly used metric for generative models.

Original: We measured the log-likelihood using AIS 2 on GANs is strange, as shown in Figure 1.
Revised: We measured the log-likelihood using AIS 2 on GANs is strange, as shown in Figure 1.

Original: We measured the log-likelihood of the DCGAN on MNIST with three different variances, σ 2 = 0 . 01 , 0 . 025 , and 0 . 05 .
Revised: We measured the log-likelihood of the DCGAN on MNIST with three different variances, σ 2 = 0 . 01 , 0 . 025 , and 0 . 05 .

Original: The ﬁgure illustrates that the log-likelihood curve over the training epochs varies substantially depending on the variance, which indicates that the ﬁxed Gaussian observable model might not be the ideal assumption for GANs.
Revised: The ﬁgure illustrates that the log-likelihood curve over the training epochs varies substantially depending on the variance, which indicates that the ﬁxed Gaussian observable model might not be the ideal assumption for GANs.

Original: Moreover, we observe a high log-likelihood at the beginning of training, followed by a drop in likelihood, which then returns to the high value.
Revised: Moreover, we observe a high log-likelihood at the beginning of training, followed by a drop in likelihood, which then returns to the high value.

Original: The IS and MMD metrics do not require training a critic.
Revised: The IS and MMD metrics do not require training a critic.

Original: It was easy to ﬁnd samples for which IS and
Revised: It was easy to ﬁnd samples for which IS and

Original: MMD scores did not match their visual quality.
Revised: MMD scores did not match their visual quality.

Original: For example, Figure 2 shows samples generated by a
Revised: For example, Figure 2 shows samples generated by a

Original: DCGAN when it failed to train properly.
Revised: DCGAN when it failed to train properly.

Original: Even though the failed DCGAN samples are much darker than the samples on the right, the IS for the left samples is
Revised: Even though the failed DCGAN samples are much darker than the samples on the right, the IS for the left samples is

Original: [Figure]
Revised: [Figure]

Original: I
Revised: I

Original: [Figure]
Revised: [Figure]

Original: p
Revised: p

Original: [Figure]
Revised: [Figure]

Original: op
Revised: op

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: As the Imagenet-trained network is likely trained to be somewhat invariant to overall intensity, this issue is to be expected.
Revised: As the Imagenet-trained network is likely trained to be somewhat invariant to overall intensity, this issue is to be expected.

Original: A failure case for MMD is shown in Figure 5.
Revised: A failure case for MMD is shown in Figure 5.

Original: The samples on the right are dark, like the previous examples, but still textually recognizable, whereas the samples on the left are totally meaningless.
Revised: The samples on the right are dark, like the previous examples, but still textually recognizable, whereas the samples on the left are totally meaningless.

Original: However, MMD gives lower/worse distances to the left samples.
Revised: However, MMD gives lower/worse distances to the left samples.

Original: The average intensity of the pixels of the left samples are closer to that for the training data, suggesting that MMD is overly sensitive to image intensity.
Revised: The average intensity of the pixels of the left samples are closer to that for the training data, suggesting that MMD is overly sensitive to image intensity.

Original: Thus, IS is under-sensitive to image intensity, while MMD if oversensitive to it.
Revised: Thus, IS is under-sensitive to image intensity, while MMD if oversensitive to it.

Original: In
Revised: In

Original: Section 4.2.1, we conduct more systematic experiments by measuring the correlation between these metrics to human perceptual scores.
Revised: Section 4.2.1, we conduct more systematic experiments by measuring the correlation between these metrics to human perceptual scores.

Original: 4. M ETRIC COMPARISON
Revised: 4. M ETRIC COMPARISON

Original: To both compare the metrics as well as different GAN frameworks, we evaluated the six metrics on different GAN frameworks.
Revised: To both compare the metrics as well as different GAN frameworks, we evaluated the six metrics on different GAN frameworks.

Original: Tables 2, 3, and 4 present the results on MNIST, CIFAR10, and LSUN respectively.
Revised: Tables 2, 3, and 4 present the results on MNIST, CIFAR10, and LSUN respectively.

Original: As each type of GAN was trained using one of our proposed metrics, we investigated whether the metric favors samples from the model trained using the same metric.
Revised: As each type of GAN was trained using one of our proposed metrics, we investigated whether the metric favors samples from the model trained using the same metric.

Original: Interestingly, we do not see this behavior, and our proposed metrics agree on which GAN framework produces samples closest to the test data set.
Revised: Interestingly, we do not see this behavior, and our proposed metrics agree on which GAN framework produces samples closest to the test data set.

Original: Every metric, except for MMD, showed that LS-DCGAN performed best for MNIST and CIFAR10, while W-DCGAN performed best for LSUN.
Revised: Every metric, except for MMD, showed that LS-DCGAN performed best for MNIST and CIFAR10, while W-DCGAN performed best for LSUN.

Original: As discussed below, we found DCGAN to be unstable to train, and thus excluded GC as a metric for experiments except for this ﬁrst data set.
Revised: As discussed below, we found DCGAN to be unstable to train, and thus excluded GC as a metric for experiments except for this ﬁrst data set.

Original: For Fashion-MNIST, FID’s ranking disagreed with IW and LS.
Revised: For Fashion-MNIST, FID’s ranking disagreed with IW and LS.

Original: We observed similar results for a range of different critic CNN architectures (number of feature maps in each convolutional layer): [3 , 64 , 128 , 256] , [3 , 128 , 256 , 512] , [3 , 256 , 512 , 1024] , and [3 , 320 , 640 , 1280] (see Supp.
Revised: We observed similar results for a range of different critic CNN architectures (number of feature maps in each convolutional layer): [3 , 64 , 128 , 256] , [3 , 128 , 256 , 512] , [3 , 256 , 512 , 1024] , and [3 , 320 , 640 , 1280] (see Supp.

Original: Fig.
Revised: Fig.

Original: 12 and 13).
Revised: 12 and 13).

Original: We evaluated a larger variety of GAN frameworks using pre-trained GANs downloaded from (pyt).
Revised: We evaluated a larger variety of GAN frameworks using pre-trained GANs downloaded from (pyt).

Original: In particular, we evaluated on EBGAN(Junbo Zhao, 2016), BEGAN(Berthelot et al., 2017), W-DCGAN
Revised: In particular, we evaluated on EBGAN(Junbo Zhao, 2016), BEGAN(Berthelot et al., 2017), W-DCGAN

Original: GP(Gulrajani et al., 2017), and DRAGAN (Kodali et al., 2017).
Revised: GP(Gulrajani et al., 2017), and DRAGAN (Kodali et al., 2017).

Original: Table 5 presents the evaluation results.
Revised: Table 5 presents the evaluation results.

Original: Critic architectures were selected to match those of these pre-trained GANs.
Revised: Critic architectures were selected to match those of these pre-trained GANs.

Original: For both MNIST and FashionMNIST, the three metrics are consistent and they rank DRAGAN the highest, followed by LS-DCGAN and DCGAN.
Revised: For both MNIST and FashionMNIST, the three metrics are consistent and they rank DRAGAN the highest, followed by LS-DCGAN and DCGAN.

Original: The standard deviations for the IW distance are higher than for LS divergence.
Revised: The standard deviations for the IW distance are higher than for LS divergence.

Original: We computed the
Revised: We computed the

Original: Wilcoxon rank sum in order to test that whether medians of the distributions of distances are the same for DCGAN, LS-DCGAN, and W-DCGAN.
Revised: Wilcoxon rank sum in order to test that whether medians of the distributions of distances are the same for DCGAN, LS-DCGAN, and W-DCGAN.

Original: We found that the different GAN frameworks have signiﬁcantly different performance according to the LS-GAN criterion, but not according to the IW criterion ( p < . 05 , Wilcoxon rank-sum test).
Revised: We found that the different GAN frameworks have signiﬁcantly different performance according to the LS-GAN criterion, but not according to the IW criterion ( p < . 05 , Wilcoxon rank-sum test).

Original: Thus LS is more sensitive than IW.
Revised: Thus LS is more sensitive than IW.

Original: We evaluated the consistency of the metrics with respect to the size of validation set.
Revised: We evaluated the consistency of the metrics with respect to the size of validation set.

Original: We trained our three GAN frameworks for 100 epochs with training 90,000 examples from the LSUN Bedroom
Revised: We trained our three GAN frameworks for 100 epochs with training 90,000 examples from the LSUN Bedroom

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: dataset.
Revised: dataset.

Original: We then trained LS and IW critics using both 300 and 90,000 validation examples.
Revised: We then trained LS and IW critics using both 300 and 90,000 validation examples.

Original: We looked at how often the critic trained with 300 examples agreed with that trained with 90,000 examples.
Revised: We looked at how often the critic trained with 300 examples agreed with that trained with 90,000 examples.

Original: The
Revised: The

Original: LS critics agreed 88% of the time, while the IW critics agreed only 55% of the time (slightly better than chance).
Revised: LS critics agreed 88% of the time, while the IW critics agreed only 55% of the time (slightly better than chance).

Original: Thus, LS is more robust to validation data set size.
Revised: Thus, LS is more robust to validation data set size.

Original: Another advantage is that measuring the LS distance is faster than measuring the IW distance, as estimating IW involves regularizing with a gradient penalty time (Gulrajani et al., 2017).
Revised: Another advantage is that measuring the LS distance is faster than measuring the IW distance, as estimating IW involves regularizing with a gradient penalty time (Gulrajani et al., 2017).

Original: Computing the gradient penalty term and tuning its regularization coefﬁcient requires extra computational time.
Revised: Computing the gradient penalty term and tuning its regularization coefﬁcient requires extra computational time.

Original: As mentioned above, we found training a critic using the GC criterion (corresponding to a DCGAN) to be unstable.
Revised: As mentioned above, we found training a critic using the GC criterion (corresponding to a DCGAN) to be unstable.

Original: It has previously been speculated that this is the case because the support of the data and model distributions possibly becoming disjoint (Arjovsky & Bottou, 2017), and the Hessian of the GAN objective being non-Hermitian (Mescheder et al., 2017).
Revised: It has previously been speculated that this is the case because the support of the data and model distributions possibly becoming disjoint (Arjovsky & Bottou, 2017), and the Hessian of the GAN objective being non-Hermitian (Mescheder et al., 2017).

Original: LS-DCGAN and W-DCGAN are proposed to address this by providing non-saturating gradients.
Revised: LS-DCGAN and W-DCGAN are proposed to address this by providing non-saturating gradients.

Original: We also found DCGAN to be difﬁcult to train, and thus only report results using the corresponding criterion GC for MNIST.
Revised: We also found DCGAN to be difﬁcult to train, and thus only report results using the corresponding criterion GC for MNIST.

Original: Note that this is different than training a discriminator as part of standard GAN training because we are training from a random initialization, not from the previous version of the discriminator.
Revised: Note that this is different than training a discriminator as part of standard GAN training because we are training from a random initialization, not from the previous version of the discriminator.

Original: Our experience was that the LS-DCGAN was the simplest and most stable model to train.
Revised: Our experience was that the LS-DCGAN was the simplest and most stable model to train.

Original: We visualized the 2D subspace of the loss surface of the GANs in Supp.
Revised: We visualized the 2D subspace of the loss surface of the GANs in Supp.

Original: Fig.
Revised: Fig.

Original: Here, we took the parameters of three trained models (corresponds to red vertices in the ﬁgure) and applied barycentric interpolation with respect to three parameters (see details from (Im et al., 2016c)).
Revised: Here, we took the parameters of three trained models (corresponds to red vertices in the ﬁgure) and applied barycentric interpolation with respect to three parameters (see details from (Im et al., 2016c)).

Original: DCGAN surfaces have much sharper slopes when compared to the LS-DCGAN and W-DCGAN, and LS-DCGAN has the most gentle surfaces.
Revised: DCGAN surfaces have much sharper slopes when compared to the LS-DCGAN and W-DCGAN, and LS-DCGAN has the most gentle surfaces.

Original: In what follows, we show that this geometric view is consistent with our ﬁnding that LS-DCGAN is the easiest and the most stable to train.
Revised: In what follows, we show that this geometric view is consistent with our ﬁnding that LS-DCGAN is the easiest and the most stable to train.

Original: [Table]
Revised: [Table]

Original: C OMPARISON TO H UMAN P ERCEPTION
Revised: C OMPARISON TO H UMAN P ERCEPTION

Original: We compared the LS, IW, MMD, and IS metrics to human perception for the CIFAR10 dataset.
Revised: We compared the LS, IW, MMD, and IS metrics to human perception for the CIFAR10 dataset.

Original: To accomplish this, we asked ﬁve volunteers to choose which of two sets of 100 samples, each generated using a different generator, looked most realistic.
Revised: To accomplish this, we asked ﬁve volunteers to choose which of two sets of 100 samples, each generated using a different generator, looked most realistic.

Original: Before surveying, the volunteers were trained to choose between real samples from CIFAR10 and samples generated by a GAN.
Revised: Before surveying, the volunteers were trained to choose between real samples from CIFAR10 and samples generated by a GAN.

Original: Supp.
Revised: Supp.

Original: Fig.displays the user interface for the participants, and Supp.
Revised: Fig.displays the user interface for the participants, and Supp.

Original: Fig.
Revised: Fig.

Original: 15 shows the fraction of labels that the volunteers agreed upon.
Revised: 15 shows the fraction of labels that the volunteers agreed upon.

Original: Table 6) presents the fraction of pairs for which each metric agrees with humans (higher is better).
Revised: Table 6) presents the fraction of pairs for which each metric agrees with humans (higher is better).

Original: IW has a slight edge over LS, and both outperform IS and MMD.
Revised: IW has a slight edge over LS, and both outperform IS and MMD.

Original: In Figure 3, we show examples in which all humans agree and metrics disagrees with human perception.
Revised: In Figure 3, we show examples in which all humans agree and metrics disagrees with human perception.

Original: All such examples are shown in Supp.
Revised: All such examples are shown in Supp.

Original: Fig.
Revised: Fig.

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: . 4. S ENSITIVITY A NALYSIS 4.3. P ERFORMANCE CHANGE WITH RESPECT TO THE SIZE OF THE NETWORK
Revised: . 4. S ENSITIVITY A NALYSIS 4.3. P ERFORMANCE CHANGE WITH RESPECT TO THE SIZE OF THE NETWORK

Original: Several works have demonstrated an improvement in performance by enlarging deep network architectures (Krizhevsky et al., 2012; Simonyan & Zisserman, 2014;
Revised: Several works have demonstrated an improvement in performance by enlarging deep network architectures (Krizhevsky et al., 2012; Simonyan & Zisserman, 2014;

Original: He et al., 2015; Huang et al., 2017).
Revised: He et al., 2015; Huang et al., 2017).

Original: Here, we investigate performance changes with respect to the width and depth of the networks.
Revised: Here, we investigate performance changes with respect to the width and depth of the networks.

Original: First, we trained three GANs with varying numbers of feature map sizes, as shown in Table 7 (a-d).
Revised: First, we trained three GANs with varying numbers of feature map sizes, as shown in Table 7 (a-d).

Original: Note that we double the number of feature maps in Table 7 for both the discriminators and generators.
Revised: Note that we double the number of feature maps in Table 7 for both the discriminators and generators.

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Table]
Revised: [Table]

Original: In Figure
Revised: In Figure

Original: [Table]
Revised: [Table]

Original: the performance of the LS score increases logarithmically as the number of feature maps is doubled.
Revised: the performance of the LS score increases logarithmically as the number of feature maps is doubled.

Original: A similar behaviour is observed in other metrics as well (see S.M. Figure 16)
Revised: A similar behaviour is observed in other metrics as well (see S.M. Figure 16)

Original: [Table] [Table]
Revised: [Table] [Table]

Original: We then analyzed the importance of size in the discriminative and generative networks.
Revised: We then analyzed the importance of size in the discriminative and generative networks.

Original: We considered two extreme feature map sizes, where we choose a small and large number of feature maps for the generator and discriminator, and vice versa (see label (e) and (f) in Table 7), and results are shown in Table 8.
Revised: We considered two extreme feature map sizes, where we choose a small and large number of feature maps for the generator and discriminator, and vice versa (see label (e) and (f) in Table 7), and results are shown in Table 8.

Original: For LS-DCGAN, it can be seen that a large number of feature maps for the discriminator has a better score than a large number of feature maps for the generator.
Revised: For LS-DCGAN, it can be seen that a large number of feature maps for the discriminator has a better score than a large number of feature maps for the generator.

Original: This can also be qualitatively veriﬁed by looking at the samples from architectures (a), (e), (f), and (d) in Figure 6.
Revised: This can also be qualitatively veriﬁed by looking at the samples from architectures (a), (e), (f), and (d) in Figure 6.

Original: For W-DCGAN, we observe the agreement between the LS and IW metric and conﬂict with MMD and IS.
Revised: For W-DCGAN, we observe the agreement between the LS and IW metric and conﬂict with MMD and IS.

Original: When we look at the samples from the W-DCGAN in Figure 5, it is clear that the model with a larger number of feature maps in the discriminator should get a better score; this is another example of false intuition propagated by MMD and IS.
Revised: When we look at the samples from the W-DCGAN in Figure 5, it is clear that the model with a larger number of feature maps in the discriminator should get a better score; this is another example of false intuition propagated by MMD and IS.

Original: One interesting observation is that when we compare the score and samples from architecture (a) and (e) from Table 7, architecture (a) is much better than (e) (see Figure 6).
Revised: One interesting observation is that when we compare the score and samples from architecture (a) and (e) from Table 7, architecture (a) is much better than (e) (see Figure 6).

Original: This demonstrates that having a large generator and small discriminator is worse than having a small architecture for both networks.
Revised: This demonstrates that having a large generator and small discriminator is worse than having a small architecture for both networks.

Original: Overall, we found that having a larger generator than discriminator does not give good results, and that it is more desirable to have a larger discriminator than generator.
Revised: Overall, we found that having a larger generator than discriminator does not give good results, and that it is more desirable to have a larger discriminator than generator.

Original: Similar results were also observed for
Revised: Similar results were also observed for

Original: MNIST, as shown in S.M. Figure 20.
Revised: MNIST, as shown in S.M. Figure 20.

Original: This result somewhat supports the theoretical result from Arora et al.
Revised: This result somewhat supports the theoretical result from Arora et al.

Original: (2017), where the generator capacity needs to be modulated in order for approximately pure equilibrium to exist for GANs.
Revised: (2017), where the generator capacity needs to be modulated in order for approximately pure equilibrium to exist for GANs.

Original: Lastly, we experimented with how performance changes with respect to the dimension of the noise vectors.
Revised: Lastly, we experimented with how performance changes with respect to the dimension of the noise vectors.

Original: The source of the sample starts by transforming a noise vector into a meaningful image.
Revised: The source of the sample starts by transforming a noise vector into a meaningful image.

Original: It is unclear how the size of noise affects the ability of the generator to generate a meaningful image.
Revised: It is unclear how the size of noise affects the ability of the generator to generate a meaningful image.

Original: Che et al.
Revised: Che et al.

Original: (2017) have observed that a 100-d noise vector preserves modes better than a 200-d noise vector
Revised: (2017) have observed that a 100-d noise vector preserves modes better than a 200-d noise vector

Original: “Small” number of ﬁlters for both discriminator and generator (ref.
Revised: “Small” number of ﬁlters for both discriminator and generator (ref.

Original: (a) in Table 7).
Revised: (a) in Table 7).

Original: “Small” and “large” number of ﬁlters for discriminator and generator respectively (ref.
Revised: “Small” and “large” number of ﬁlters for discriminator and generator respectively (ref.

Original: (e) in
Revised: (e) in

Original: Table 7).
Revised: Table 7).

Original: “Large” and “small” number of ﬁlters for discriminator and generator respectively (ref.
Revised: “Large” and “small” number of ﬁlters for discriminator and generator respectively (ref.

Original: (f) in
Revised: (f) in

Original: Table 7).
Revised: Table 7).

Original: “Large” number of ﬁlters for both discriminator and generator (ref.
Revised: “Large” number of ﬁlters for both discriminator and generator (ref.

Original: (d) in Table 7).
Revised: (d) in Table 7).

Original: for DCGAN.
Revised: for DCGAN.

Original: Our experiments show that this depends on the model.
Revised: Our experiments show that this depends on the model.

Original: Given a ﬁxed size architecture (d) from Table 7, we observed the performance of LS-DCGAN and W-DCGAN by varying the size of noise vector z .
Revised: Given a ﬁxed size architecture (d) from Table 7, we observed the performance of LS-DCGAN and W-DCGAN by varying the size of noise vector z .

Original: Table 9 illustrates that LS-DCGAN gives the best score with a noise dimension ofand W-DCGAN gives best score with a noise dimension of 150 for both IW and LS.
Revised: Table 9 illustrates that LS-DCGAN gives the best score with a noise dimension ofand W-DCGAN gives best score with a noise dimension of 150 for both IW and LS.

Original: The outcome of
Revised: The outcome of

Original: LS-DCGAN is consistent with the result in (Che et al., 2017).
Revised: LS-DCGAN is consistent with the result in (Che et al., 2017).

Original: It is possible that this occurs because both models fall into the category of f -divergences, whereas the W-DCGAN behaves differently because its metric falls under a different category, the Integral Probability Metric
Revised: It is possible that this occurs because both models fall into the category of f -divergences, whereas the W-DCGAN behaves differently because its metric falls under a different category, the Integral Probability Metric

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Figure]
Revised: [Figure]

Original: 4.3. P ERFORMANCE CHANGE WITH RESPECT TO THE RATIO OF NUMBER OF UPDATES BETWEEN THE GENERATOR AND DISCRIMINATOR
Revised: 4.3. P ERFORMANCE CHANGE WITH RESPECT TO THE RATIO OF NUMBER OF UPDATES BETWEEN THE GENERATOR AND DISCRIMINATOR

Original: In practice, we alternate between updating the discriminator and generator
Revised: In practice, we alternate between updating the discriminator and generator

Original: [Figure]
Revised: [Figure]

Original: and yet this is not guaranteed to give the same result as the solution to the min-max problem in Equation 2.
Revised: and yet this is not guaranteed to give the same result as the solution to the min-max problem in Equation 2.

Original: Hence, the update ratio can inﬂuence the performance of GANs.
Revised: Hence, the update ratio can inﬂuence the performance of GANs.

Original: We experimented with three different update ratios, 5 : 1 , 1 :
Revised: We experimented with three different update ratios, 5 : 1 , 1 :

Original: [Figure]
Revised: [Figure]

Original: and 1 : 5 , with respect to the discriminator and generator update.
Revised: and 1 : [Figure] with respect to the discriminator and generator update.

Original: We applied these ratios to both the MNIST and CIFAR10 datasets on all models.
Revised: We applied these ratios to both the MNIST and CIFAR10 datasets on all models

Revised: [Figure]

Original: Figure 7 presents the LS scores on both MNIST and CIFAR10 and this result is consistent with the
Revised: Figure 7 presents the LS scores on both MNIST and CIFAR10 and this result is consistent with the

Original: IW metric as well (see S.M. Figure 25).
Revised: IW metric as well (see S.M. Figure 25).

Original: However, we did not ﬁnd that any one update ratio was superior over others between the two datasets.
Revised: However, we did not ﬁnd that any one update ratio was superior over others between the two datasets.

Original: For CIFAR10, the 1 : 1 update ratio worked best for all models, and for MNIST, different ratios worked better for different models.
Revised: For CIFAR10, the 1 : 1 update ratio worked best for all models, and for MNIST, different ratios worked better for different models.

Original: Hence, we conclude that number of update ratios for each model needs to be dynamically tuned.
Revised: Hence, we conclude that number of update ratios for each model needs to be dynamically tuned.

Original: The corresponding samples from the models trained by different update ratios are shown in S.M. Figure 27.
Revised: The corresponding samples from the models trained by different update ratios are shown in S.M. Figure 27.

Original: 4.3. P ERFORMANCE WITH RESPECT TO THE AMOUNT OF AVAILABLE TRAINING DATA
Revised: 4.3. P ERFORMANCE WITH RESPECT TO THE AMOUNT OF AVAILABLE TRAINING DATA

Original: [Figure]
Revised: [Figure]

Original: In practice, DCGANs are known to be unstable, and the generator tends to suffer as the discriminator gets better due to disjoint support between the data and generator distributions (Goodfellow et al., 2014; Arjovsky & Bottou, 2017).
Revised: In practice, DCGANs are known to be unstable, and the generator tends to suffer as the discriminator gets better due to disjoint support between the data and generator distributions (Goodfellow et al., 2014; Arjovsky & Bottou, 2017).

Original: W-DCGAN and LS-DCGAN offer alternative ways to solving this problem.
Revised: W-DCGAN and LS-DCGAN offer alternative ways to solving this problem.

Original: If the model is suffering from disjoint support, having more training examples will not help, and alternatively, if the model does not suffer from such a problem, having more training examples could potentially help.
Revised: If the model is suffering from disjoint support, having more training examples will not help, and alternatively, if the model does not suffer from such a problem, having more training examples could potentially help.

Original: Here, we explore the sensitivity of three different kinds of GANs with respect to the number of training examples.
Revised: Here, we explore the sensitivity of three different kinds of GANs with respect to the number of training examples.

Original: We have trained GANs with 10,000, 20,000, 30,000, 40,000, and 45,000 examples on CIFAR10.
Revised: We have trained GANs with 10,000, 20,000, 30,000, 40,000, and 45,000 examples on CIFAR10.

Original: Figure 8 shows that the LS score curve of DCGAN grows quite slowly when compared to W-DCGAN and LS-DCGAN.
Revised: Figure 8 shows that the LS score curve of DCGAN grows quite slowly when compared to W-DCGAN and LS-DCGAN.

Original: The three
Revised: The three

Original: GANs have a relatively similar loss when they are trained with 10,000 training examples.
Revised: GANs have a relatively similar loss when they are trained with 10,000 training examples.

Original: However, the DCGAN only gained 0 .
Revised: However, the DCGAN only gained 0 .

Original: 0124 ± 0 . 

Revised: 0124 ± 0 .

Original: by increasing from 10,000 to 40,000 training examples, whereas the performance of W-DCGAN and LS-DCGAN improved by 0 .
Revised: by increasing from 10,000 to 40,000 training examples, whereas the performance of W-DCGAN and LS-DCGAN improved by 0 .

Original: 03016 ± 0 . 00469 and 0 . 0444 ± 0 . 0033 , respectively.
Revised: 03016 ± 0 . 00469 and 0 . 0444 ± 0 . 0033 , respectively.

Original: Thus, we empirically observe that W-DCGAN and LS-DCGAN have faster performance increases than a DCGAN as the number of training examples grows.
Revised: Thus, we empirically observe that W-DCGAN and LS-DCGAN have faster performance increases than a DCGAN as the number of training examples grows.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we proposed to use four well-known distance functions as an evaluation metrics, and empirically investigated the DCGAN, W-DCGAN, and LS-DCGAN families under these metrics.
Revised: In this paper, we proposed to use four well-known distance functions as an evaluation metrics, and empirically investigated the DCGAN, W-DCGAN, and LS-DCGAN families under these metrics.

Original: Previously, these models were compared based on visual assessment of sample quality and difﬁculty of training.
Revised: Previously, these models were compared based on visual assessment of sample quality and difﬁculty of training.

Original: In our experiments, we showed that there are performance differences in terms of average experiments, but that some are not statistically signiﬁcant.
Revised: In our experiments, we showed that there are performance differences in terms of average experiments, but that some are not statistically signiﬁcant.

Original: Moreover, we thoroughly analyzed the performance of GANs under different hyper-parameter settings.
Revised: Moreover, we thoroughly analyzed the performance of GANs under different hyper-parameter settings.

Original: There are still several types of GANs that need to be evaluated, such as GRAN (Im et al., 2016a), IWDCGAN (Gulrajani et al., 2017), BEGAN (Berthelot et al., 2017), MMDGAN (Li et al., 2017), and
Revised: There are still several types of GANs that need to be evaluated, such as GRAN (Im et al., 2016a), IWDCGAN (Gulrajani et al., 2017), BEGAN (Berthelot et al., 2017), MMDGAN (Li et al., 2017), and

Original: CramerGAN (Bellemare et al., 2017).
Revised: CramerGAN (Bellemare et al., 2017).

Original: We hope to evaluate all of these models under this framework and thoroughly analyze them in the future.
Revised: We hope to evaluate all of these models under this framework and thoroughly analyze them in the future.

Original: Moreover, there has been an investigation into taking ensemble approaches to GANs, such as Generative Adversarial Parallelization Im et al.
Revised: Moreover, there has been an investigation into taking ensemble approaches to GANs, such as Generative Adversarial Parallelization Im et al.

Original: (2016b).
Revised: (2016b).

Original: Ensemble approaches have been empirically shown to work well in many domains of research, so it would be interesting to ﬁnd out whether ensembles can also help in min-max problems.
Revised: Ensemble approaches have been empirically shown to work well in many domains of research, so it would be interesting to ﬁnd out whether ensembles can also help in min-max problems.

Original: Alternatively, we can also try to evaluate other log-likelihood-based models like NVIL (Mnih & Gregor, 2014), VAE (Kingma & Welling, 2014), DVAE (Im et al., 2015), DRAW (Gregor et al., 2015), RBMs (Hinton et al., 2006; Salakhutdinov & Hinton, 2009), NICE Dinh et al.
Revised: Alternatively, we can also try to evaluate other log-likelihood-based models like NVIL (Mnih & Gregor, 2014), VAE (Kingma & Welling, 2014), DVAE (Im et al., 2015), DRAW (Gregor et al., 2015), RBMs (Hinton et al., 2006; Salakhutdinov & Hinton, 2009), NICE Dinh et al.

Original: (2014), etc.
Revised: (2014), etc.

Original: Model evaluation is an important and complex topic.
Revised: Model evaluation is an important and complex topic.

Original: Model selection, model design, and even research direction can change depending on the evaluation metric.
Revised: Model selection, model design, and even research direction can change depending on the evaluation metric.

Original: Thus, we need to continuously explore different metrics and rigorously evaluate new models.
Revised: Thus, we need to continuously explore different metrics and rigorously evaluate new models.

Original: A PPENDIX R ELATIONSHIP BETWEEN METRICS AND BINARY CLASSIFICATION
Revised: A PPENDIX R ELATIONSHIP BETWEEN METRICS AND BINARY CLASSIFICATION

Original: In this paper, we considered four distance metrics that belong to two class of metrics, φ -divergence and IPMs.
Revised: In this paper, we considered four distance metrics that belong to two class of metrics, φ -divergence and IPMs.

Original: Sriperumbudur et al.
Revised: Sriperumbudur et al.

Original: (2009) have shown that the optimal risk function is associated with a binary classiﬁer with P and Q distributions conditioned on a class when the discriminant function is restricted to certain F (Theorem 17 from (Sriperumbudur et al., 2009)).
Revised: (2009) have shown that the optimal risk function is associated with a binary classiﬁer with P and Q distributions conditioned on a class when the discriminant function is restricted to certain F (Theorem 17 from (Sriperumbudur et al., 2009)).

Original: Let the optimal risk function be:
Revised: Let the optimal risk function be:

Original: [Equation]
Revised: [Equation]

Original: where F is the set of discriminant functions (classiﬁer), y ∈ − 1 , 1 , and L is the loss function.
Revised: where F is the set of discriminant functions (classiﬁer), y ∈ − 1 , 1 , and L is the loss function.

Original: By following derivation, we can see that the optimal risk function becomes IPM:
Revised: By following derivation, we can see that the optimal risk function becomes IPM:

Original: [Equation]
Revised: [Equation]

Original: where L (1 , f ( x ))
Revised: where L (1 , f ( x ))

Original: = 1 /(cid:15) and L (0 , f ( x ))
Revised: = 1 /(cid:15) and L (0 , f ( x ))

Original: = − 1 / (1 − (cid:15) ) .
Revised: = − 1 / (1 − (cid:15) ) .

Original: The second equality is derived by separating the loss for class 1 and class 0.
Revised: The second equality is derived by separating the loss for class 1 and class 0.

Original: The third equality is from the way how we chose L(1,f(x)) and L(0,f(x)).
Revised: The third equality is from the way how we chose L(1,f(x)) and L(0,f(x)).

Original: The last equality is derived from that fact that F is symmetric around zero ( f ∈ F = > − f ∈ F ) .
Revised: The last equality is derived from that fact that F is symmetric around zero ( f ∈ F = > − f ∈ F ) .

Original: Hence, this shows that with appropriately choosing
Revised: Hence, this shows that with appropriately choosing

Original: L , MMD and Wasserstein distance can be understood as the optimal L -risk associated with binary classiﬁer with speciﬁc set of F functions.
Revised: L , MMD and Wasserstein distance can be understood as the optimal L -risk associated with binary classiﬁer with speciﬁc set of F functions.

Original: For example, Wasserstein distance and MMD distances are equivalent to the optimal risk function with 1-Lipschitz classiﬁers and a RKHS classiﬁer with an unit length.
Revised: For example, Wasserstein distance and MMD distances are equivalent to the optimal risk function with 1-Lipschitz classiﬁers and a RKHS classiﬁer with an unit length.

Original: E XPERIMENTAL H YPER - PARAMETERS
Revised: E XPERIMENTAL H YPER - PARAMETERS

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: W
Revised: W

Original: [Figure]
Revised: [Figure]

Original: DCGAN 0.
Revised: DCGAN 0.

Original: LS
Revised: LS

Original: [Figure]
Revised: [Figure]

Original: Figure
Revised: Figure

Original: W
Revised: W

Original: [Figure]
Revised: [Figure]

Original: LS
Revised: LS

Original: [Figure]
Revised: [Figure]

Original: [3, 128, 256, 512] 11 W
Revised: [3, 128, 256, 512] 11 W

Original: [Figure]
Revised: [Figure]

Original: DCGAN 0.
Revised: DCGAN 0.

Original: LS
Revised: LS

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [3, 256, 512, 1028] 11 W
Revised: [3, 256, 512, 1028] 11 W

Original: DCGAN 0.
Revised: DCGAN 0.

Original: LS
Revised: LS

Original: [Figure]
Revised: [Figure]

Original: DCGAN 0.
Revised: DCGAN 0.

Original: M ORE E XPERIMENTS
Revised: M ORE E XPERIMENTS

Original: We trained two critics on training data and validation data, respectively, and evaluated on test data from both critics.
Revised: We trained two critics on training data and validation data, respectively, and evaluated on test data from both critics.

Original: We trained six GANs (GAN, LS-DCGAN, W-DCGAN GP, DRAGAN, BEGAN,
Revised: We trained six GANs (GAN, LS-DCGAN, W-DCGAN GP, DRAGAN, BEGAN,

Original: EBGAN) on MNIST and FashionMNIST.
Revised: EBGAN) on MNIST and FashionMNIST.

Original: We trained these GANs with 50,000 training examples.
Revised: We trained these GANs with 50,000 training examples.

Original: At test time, we used 10,000 training and 10,000 validation examples for training the critics, and evaluated on 10,000 test examples.
Revised: At test time, we used 10,000 training and 10,000 validation examples for training the critics, and evaluated on 10,000 test examples.

Original: Here, we present the test scores from the critics trained on training and validation data.
Revised: Here, we present the test scores from the critics trained on training and validation data.

Original: The results are shown in Table ?? .
Revised: The results are shown in Table ?? .

Original: Note that we also have the IW and
Revised: Note that we also have the IW and

Original: FID evaluation on these models in the paper.
Revised: FID evaluation on these models in the paper.

Original: For FashionMNIST, we ﬁnd that test scores with a critic trained on training and validation data are very close.
Revised: For FashionMNIST, we ﬁnd that test scores with a critic trained on training and validation data are very close.

Original: Hence, we do not see any indication of overﬁtting.
Revised: Hence, we do not see any indication of overﬁtting.

Original: On the other hand, there are gaps between the scores for the MNIST dataset and the test scores from critics trained on the validation set.
Revised: On the other hand, there are gaps between the scores for the MNIST dataset and the test scores from critics trained on the validation set.

Original: which gives better performance than the ones that are trained on the training set.
Revised: which gives better performance than the ones that are trained on the training set.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: are evaluation under Wasserstein distance.
Revised: are evaluation under Wasserstein distance.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: i v e r g e n c e
Revised: i v e r g e n c e

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: Human: 3, 0, MMD: 0.13, 0.
Revised: Human: 3, 0, MMD: 0.13, 0.

Original: Human: 3, 0, MMD: 1.01, 0.
Revised: Human: 3, 0, MMD: 1.01, 0.

Original: Human: 3, 0, MMD: 1.20, 0. Human: 3, 0, MMD: 1.20, 1.
Revised: Human: 3, 0, MMD: 1.20, 0. Human: 3, 0, MMD: 1.20, 1.

Original: Human: 3, 0, MMD: 1.14, 0.
Revised: Human: 3, 0, MMD: 1.14, 0.

Original: Human: 3, 0, MMD: 1.56, 1.
Revised: Human: 3, 0, MMD: 1.56, 1.

Original: Human: 3, 0, MMD: 0.09, 0.
Revised: Human: 3, 0, MMD: 0.09, 0.

Original: Human: 3, 0, MMD: 1.14, 0.
Revised: Human: 3, 0, MMD: 1.14, 0.

Original: Human: 3, 0, MMD: 1.15, 1.
Revised: Human: 3, 0, MMD: 1.15, 1.

Original: Human: 4, 0, MMD: 1.08, 0.
Revised: Human: 4, 0, MMD: 1.08, 0.

Original: Human: 3, 0, MMD: 1.28, 0.
Revised: Human: 3, 0, MMD: 1.28, 0.

Original: Human: 3, 0, MMD: 1.19, 0.
Revised: Human: 3, 0, MMD: 1.19, 0.

Original: Human: 3, 0, MMD: 1.73, 0.
Revised: Human: 3, 0, MMD: 1.73, 0.

Original: Human: 3, 0, MMD: 0.61, 0.
Revised: Human: 3, 0, MMD: 0.61, 0.

Original: Human: 3, 0, MMD: 0.80, 0.
Revised: Human: 3, 0, MMD: 0.80, 0.

Original: Human: 3, 0, MMD: 1.12, 1.
Revised: Human: 3, 0, MMD: 1.12, 1.

Original: Human: 3, 0, MMD: 0.90, 0.
Revised: Human: 3, 0, MMD: 0.90, 0.

Original: Human: 3, 0, MMD: 0.04, 0.
Revised: Human: 3, 0, MMD: 0.04, 0.

Original: Human: 3, 0, MMD: 0.01, 0.
Revised: Human: 3, 0, MMD: 0.01, 0.

Original: Human: 4, 1.000000e+00, MMD: 0.11, 0.
Revised: Human: 4, 1.000000e+00, MMD: 0.11, 0.

Original: Human: 3, 0, MMD: 0.32, 0.
Revised: Human: 3, 0, MMD: 0.32, 0.

Original: Human: 3, 0, MMD: 0.01, 0.
Revised: Human: 3, 0, MMD: 0.01, 0.

Original: [Figure]
Revised: [Figure]

Original: [Figure]

Original: [Figure]

Original: [Figure]
Revised: [Figure]

Original: W-DCGAN [Figure] [Figure] Ratio
Revised: Performance of DCGAN, W

Original: LS
Revised: DCGAN, and LS-DCGAN trained with varying of

Original: [Figure]
Revised: [Figure]

Original: DCGAN

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure][Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure][Figure]
Revised: [Figure] [Figure]