Original: R ecycling the D iscriminator for I mproving the I nference M apping of GAN
Revised: R ecycling the D iscriminator for I mproving the I nference M apping of GAN

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A bstract
Revised: A bstract

Original: Generative adversarial networks (GANs) have achieved outstanding success in generating the high-quality data.
Revised: Generative adversarial networks (GANs) have achieved outstanding success in generating the high-quality data.

Original: Focusing on the generation process, existing GANs learn a unidirectional mapping from the latent vector to the data.
Revised: Focusing on the generation process, existing GANs learn a unidirectional mapping from the latent vector to the data.

Original: Later, various studies point out that the latent space of GANs is semantically meaningful and can be utilized for advanced data analysis and manipulation.
Revised: Later, various studies point out that the latent space of GANs is semantically meaningful and can be utilized for advanced data analysis and manipulation.

Original: In order to analyze the real data in the latent space of GANs, it is necessary to investigate the inverse generation mapping from the data to the latent vector.
Revised: In order to analyze the real data in the latent space of GANs, it is necessary to investigate the inverse generation mapping from the data to the latent vector.

Original: To tackle this problem, the bidirectional generative models introduce an encoder to establish the inverse path of the generation process.
Revised: To tackle this problem, the bidirectional generative models introduce an encoder to establish the inverse path of the generation process.

Original: Unfortunately, their inference mapping does not accurately predict the latent vector from the data because the imperfect generator rather interferes the encoder training.
Revised: Unfortunately, their inference mapping does not accurately predict the latent vector from the data because the imperfect generator rather interferes the encoder training.

Original: In this paper, we propose an e ﬀ ective algorithm to accurately infer the latent vector based on existing unidirectional GANs.
Revised: In this paper, we propose an e ﬀ ective algorithm to accurately infer the latent vector based on existing unidirectional GANs.

Original: It is important to note that we focus on increasing the accuracy and e ﬃ ciency of the inference mapping but not inﬂuencing the GAN performance toward image quality or diversity.
Revised: It is important to note that we focus on increasing the accuracy and e ﬃ ciency of the inference mapping but not inﬂuencing the GAN performance toward image quality or diversity.

Original: Instead, utilizing the proposed inference mapping algorithm, we suggest a new metric for evaluating the GAN models by measuring the reconstruction error of unseen real data.
Revised: Instead, utilizing the proposed inference mapping algorithm, we suggest a new metric for evaluating the GAN models by measuring the reconstruction error of unseen real data.

Original: The experimental analysis demonstrates that the proposed algorithm achieves more accurate inference mapping than existing methods and provides the robust metric for evaluating GAN performance.
Revised: The experimental analysis demonstrates that the proposed algorithm achieves more accurate inference mapping than existing methods and provides the robust metric for evaluating GAN performance.

Original:  I ntroduction
Revised:  I ntroduction

Original: Generative adversarial networks (GANs) have reported a remarkable progress for successfully reproducing the real data distribution, particularly natural images.
Revised: Generative adversarial networks (GANs) have reported a remarkable progress for successfully reproducing the real data distribution, particularly natural images.

Original: Although GANs imposes few constraint or assumption on their model deﬁnition, even without the variational bound, it is capable of producing sharp and realistic images.
Revised: Although GANs imposes few constraint or assumption on their model deﬁnition, even without the variational bound, it is capable of producing sharp and realistic images.

Original: To this end, training the GANs involves the adversarial competition between a generator and a discriminator; the generator learns the generation process formulated by mapping the latent distribution P z to the data distribution P data ; the discriminator evaluates the generation quality by distinguishing generated images from real images.
Revised: To this end, training the GANs involves the adversarial competition between a generator and a discriminator; the generator learns the generation process formulated by mapping the latent distribution P z to the data distribution P data ; the discriminator evaluates the generation quality by distinguishing generated images from real images.

Original: Goodfellow et al.
Revised: Goodfellow et al.

Original: (2014) formulates the objective of this adversarial training using the following minimax game:
Revised: (2014) formulates the objective of this adversarial training using the following minimax game:

Original: [Equation]
Revised: [Equation]

Original: where E denotes expectation, G and D are the generator and the discriminator respectively, and x and z are samples drawn from P z and P data respectively.
Revised: where E denotes expectation, G and D are the generator and the discriminator respectively, and x and z are samples drawn from P z and P data respectively.

Original: Once the generator learns the mapping from the latent to the data distribution, it is possible to generate arbitrary data corresponding to randomly drawn z .
Revised: Once the generator learns the mapping from the latent to the data distribution, it is possible to generate arbitrary data corresponding to randomly drawn z .

Original: Because the generator network never observes the real data directly during training, it does not memorize the training dataset, thus produces unseen data.
Revised: Because the generator network never observes the real data directly during training, it does not memorize the training dataset, thus produces unseen data.

Original: Since this pioneer work, various GAN models have been developed to improve the training stability, the image quality, or the diversity of generation process.
Revised: Since this pioneer work, various GAN models have been developed to improve the training stability, the image quality, or the diversity of generation process.

Original: Recent studies ﬁnd that the latent space of GANs derives the semantically meaningful representation (Mikolov et al., 2013).
Revised: Recent studies ﬁnd that the latent space of GANs derives the semantically meaningful representation (Mikolov et al., 2013).

Original: Beneﬁt from its semantic power, several studies (Radford et al., 2016; Berthelot et al., 2017) show that it is possible to utilize the latent space of GANs for data augmentation or image editing.
Revised: Beneﬁt from its semantic power, several studies (Radford et al., 2016; Berthelot et al., 2017) show that it is possible to utilize the latent space of GANs for data augmentation or image editing.

Original: To further understand and interpret the semantic representation derived by the latent space of GANs, we should investigate the inference mapping from x to z .
Revised: To further understand and interpret the semantic representation derived by the latent space of GANs, we should investigate the inference mapping from x to z .

Original: Learning this inference mapping can be formulated as minimizing the reconstruction error by improving the latent estimation.
Revised: Learning this inference mapping can be formulated as minimizing the reconstruction error by improving the latent estimation.

Original:  BEGAN (Berthelot et al., 2017) madethe ﬁrst attempt to solve the inverse mapping from x to z using the non-convex optimization.
Revised: CoGAN (Liu & Tuzel, 2016) and BEGAN (Berthelot et al., 2017) made attempt to solve the inverse mapping from x to z using the non-convex optimization.

Original: More speciﬁcally, the problem can be deﬁned as min z d (x , G ( z )) , where d( · ) is the distance metric.
Revised: More speciﬁcally, the problem can be deﬁned as min z d (x , G ( z )) , where d( · ) is the distance metric.

Original: They used this inverse mapping to prove that the generated images are not the result of data memorization.

Original: It is important to note that this optimization aims to ﬁnd the inverse generation path G − 1 ( x ) using the non-convex optimizer.
Revised: It is important to note that this optimization aims to ﬁnd the inverse generation path G − 1 ( x ) using the non-convex optimizer.

Original: Because of the non-linearity and model complexity of generator, calculating the inverse path su ﬀ ers from multiple local minima, thus hard to reach the global optimum.
Revised: Because of the non-linearity and model complexity of generator, calculating the inverse path su ﬀ ers from multiple local minima, thus hard to reach the global optimum.

Original: Also, it is impractical due to its computational complexity at runtime.
Revised: Also, it is impractical due to its computational complexity at runtime.

Revised: To reduce the computational complexity and improve the accuracy of inference mapping, iGAN (Zhu et al., 2016)

Revised: ﬁrst suggest a hybrid method; they predict the initial latent vector of x using a naïve encoder and then perform the non-convex optimization to compute the best estimate of z by minimizing the pixel di ﬀ erence between G( z ) and x .

Original: Several studies pay attention to simultaneously learning the inference (i.e., from x to z ) and the generation path (i.e., from z to x ).
Revised: Several studies pay attention to simultaneously learning the inference (i.e., from x to z ) and the generation path (i.e., from z to x ).

Original: The core idea of these approaches is to employ an encoder to the GAN models for achieving the bidirectional mapping between z and x .
Revised: The core idea of these approaches is to employ an encoder to the GAN models for achieving the bidirectional mapping between z and x .

Original: Dumoulin et al.
Revised: Dumoulin et al.

Original: Donahue et al.
Revised: Donahue et al.

Original: (2017) propose novel GAN frameworks for establishing the bidirectional mapping between P z and P data jointly learnt by adversarial training.
Revised: (2017) propose novel GAN frameworks for establishing the bidirectional mapping between P z and P data jointly learnt by adversarial training.

Original: Both methods can be used to infer the latent vector of real data and alleviate the mode collapse problem.
Revised: Both methods can be used to infer the latent vector of real data and alleviate the mode collapse problem.

Original: However, their performance is relatively poor in terms of either generation quality or inference accuracy; the poor generation leads to blurry images and the poor inference results in inaccurate inference mapping.
Revised: However, their performance is relatively poor in terms of either generation quality or inference accuracy; the poor generation leads to blurry images and the poor inference results in inaccurate inference mapping.

Original: Alternatively, Variational Autoencoder (VAE) (Kingma & Welling, 2013) and Adversarial Autoencoder (AAE) (Makhzani et al., 2016) are the most representative generative models that explicitly learn the bidirectional mapping between z and x .
Revised: Alternatively, Variational Autoencoder (VAE) (Kingma & Welling, 2013) and Adversarial Autoencoder (AAE) (Makhzani et al., 2016) are the most representative generative models that explicitly learn the bidirectional mapping between z and x .

Original: Their model architectures are quite similar to the structure of autoencoder (Baldi, 2012), composed of an encoder (i.e., the inverse generator) and a decoder (i.e., the generator).
Revised: Their model architectures are quite similar to the structure of autoencoder (Baldi, 2012), composed of an encoder (i.e., the inverse generator) and a decoder (i.e., the generator).

Original: Unlike autoencoder, VAE and AAE enforce to match the latent distributions to prior distributions, thus enabling the data generation.
Revised: Unlike autoencoder, VAE and AAE enforce to match the latent distributions to prior distributions, thus enabling the data generation.

Original: While VAE utilizes KL divergence to match the latent to the target distribution, AAE utilizes the adversarial learning for latent distribution matching.
Revised: While VAE utilizes KL divergence to match the latent to the target distribution, AAE utilizes the adversarial learning for latent distribution matching.

Original: Although both algorithms establish the bidirectional mapping between the latent and the data distribution as well as training stability, their image quality is relatively worse than those of unidirectional GANs.
Revised: Although both algorithms establish the bidirectional mapping between the latent and the data distribution as well as training stability, their image quality is relatively worse than those of unidirectional GANs.

Original: Speciﬁcally, their generated images are blurry and the details are lost.
Revised: Speciﬁcally, their generated images are blurry and the details are lost.

Original: In this paper, we propose an e ﬀ ective algorithm to infer the latent vector based on existing unidirectional GANs by maintaining their generation quality.
Revised: In this paper, we propose an e ﬀ ective algorithm to infer the latent vector based on existing unidirectional GANs by maintaining their generation quality.

Original: Our goal is to improve the accuracy and e ﬃ ciency of inference mapping better than other inference mapping techniques.
Revised: Our goal is to improve the accuracy and e ﬃ ciency of inference mapping better than other inference mapping techniques.

Original: Our idea is motivated by the fact that the discriminator of GANs can serve as a meaningful feature extractor for both real and fake images.
Revised: Our idea is motivated by the fact that the discriminator of GANs can serve as a meaningful feature extractor for both real and fake images.

Original: From this intuition, we utilize the discriminator as the feature extractor for inference mapping.
Revised: From this intuition, we utilize the discriminator as the feature extractor for inference mapping.

Original: By ﬁxing the feature extractor, training our inference model is to learn the mapping from the feature to the latent vector with much fewer model parameters.
Revised: By ﬁxing the feature extractor, training our inference model is to learn the mapping from the feature to the latent vector with much fewer model parameters.

Original: We claim that our idea of recycling discriminator’s features is more powerful than building new feature extractors using encoders.
Revised: We claim that our idea of recycling discriminator’s features is more powerful than building new feature extractors using encoders.

Original: Learning the new feature extractor associates either fake images or real images, but not both.
Revised: Learning the new feature extractor associates either fake images or real images, but not both.

Original: Because the GAN discriminator observes both real and fake images for classiﬁcation, its features should be representable for both real and fake images.
Revised: Because the GAN discriminator observes both real and fake images for classiﬁcation, its features should be representable for both real and fake images.

Original: As our inference mapping utilizes these discriminator features, our algorithm is equally e ﬀ ective to the inference of both real and fake images.
Revised: As our inference mapping utilizes these discriminator features, our algorithm is equally e ﬀ ective to the inference of both real and fake images.

Original: Besides, the proposed inference mapping performs the feature-to-latent translation in a much lower dimensional space than the image-to-latent translation of encoder-based approaches, thus this helps reduce the number of model parameters and lead the e ﬃ cient model training.
Revised: Besides, the proposed inference mapping performs the feature-to-latent translation in a much lower dimensional space than the image-to-latent translation of encoder-based approaches, thus this helps reduce the number of model parameters and lead the e ﬃ cient model training.

Original: As mentioned above, the proposed method focuses on improving e ﬃ ciency and accuracy for deriving inference mapping compared to existing methods.
Revised: As mentioned above, the proposed method focuses on improving e ﬃ ciency and accuracy for deriving inference mapping compared to existing methods.

Original: For that, we learn the inference mapping independently from baseline, unidirectional GAN training.
Revised: For that, we learn the inference mapping independently from baseline, unidirectional GAN training.

Original: Note that we do not aim to improve generation quality or diversity as existing GAN studies.
Revised: Note that we do not aim to improve generation quality or diversity as existing GAN studies.

Original: Instead, we introduce an e ﬃ cient inference mapping algorithm with two major applications: 1) manipulating the image by disentangling the latent space and 2) suggesting a new metric for assessing the GAN model by measuring reconstruction errors of real data.
Revised: Instead, we introduce an e ﬃ cient inference mapping algorithm with two major applications: 1) manipulating the image by disentangling the latent space and 2) suggesting a new metric for assessing the GAN model by measuring reconstruction errors of real data.

Original:  R elated work
Revised:  R elated work

Original: Existing generative models for inducing inference mapping can be categorized into two groups.
Revised: Existing generative models for inducing inference mapping can be categorized into two groups.

Original: The ﬁrst group utilizes the structure of an autoencoder and the other group develops the joint training scheme for learning bidirectional mapping.
Revised: The ﬁrst group utilizes the structure of an autoencoder and the other group develops the joint training scheme for learning bidirectional mapping.

Original: 2.1 E ncoder - decoder architecture
Revised: 2.1 E ncoder - decoder architecture

Original: Similar to the architecture of autoencoder, VAE (Kingma & Welling, 2013) consists of the encoder for transforming the input data to the latent vector and the decoder for reconstructing the input data from the latent representation.
Revised: Similar to the architecture of autoencoder, VAE (Kingma & Welling, 2013) consists of the encoder for transforming the input data to the latent vector and the decoder for reconstructing the input data from the latent representation.

Original: While the autoencoder is designed for the dimensionality reduction,
Revised: While the autoencoder is designed for the dimensionality reduction,

Original: VAE is capable of generating samples as well by ﬁtting the latent distribution to the prior distribution (Wainwright et al., 2008).
Revised: VAE is capable of generating samples as well by ﬁtting the latent distribution to the prior distribution (Wainwright et al., 2008).

Original: Because the decoder learns a mapping from the prior distribution to the data distribution during training, the trained decoder serves as the generator.
Revised: Because the decoder learns a mapping from the prior distribution to the data distribution during training, the trained decoder serves as the generator.

Original: The weakness of VAE is that the generated images from VAE often exhibit blurs and the lack of diversity; generated images are similar to those in the training dataset.
Revised: The weakness of VAE is that the generated images from VAE often exhibit blurs and the lack of diversity; generated images are similar to those in the training dataset.

Original: It is because the objective of the decoder is to minimize the reconstruction errors.
Revised: It is because the objective of the decoder is to minimize the reconstruction errors.

Original: Nevertheless, VAE is an attractive generative model due to its e ﬃ cient and stable training.
Revised: Nevertheless, VAE is an attractive generative model due to its e ﬃ cient and stable training.

Original: 2.2 B idirectional GAN s
Revised: 2.2 B idirectional GAN s

Original: ALI (Dumoulin et al., 2017) and BiGAN (Donahue et al., 2017) suggest a new theoretical framework for training the bidirectional GANs, that jointly learn the bidirectional mapping between P z and
Revised: ALI (Dumoulin et al., 2017) and BiGAN (Donahue et al., 2017) suggest a new theoretical framework for training the bidirectional GANs, that jointly learn the bidirectional mapping between P z and

Original: P data in an unsupervised manner.
Revised: P data in an unsupervised manner.

Original: They use the generator similar to unidirectional GANs (Radford et al., 2016; Mao et al., 2017; Gulrajani et al., 2017; Warde-Farley & Bengio, 2017) for constructing the forward mapping from P z to P data , and then use the encoder to model the inference mapping from P data to P z .
Revised: They use the generator similar to unidirectional GANs (Radford et al., 2016; Mao et al., 2017; Gulrajani et al., 2017; Warde-Farley & Bengio, 2017) for constructing the forward mapping from P z to P data , and then use the encoder to model the inference mapping from P data to P z .

Original: To train the generator and the encoder simultaneously, they deﬁne a new objective function for the discriminator, which distinguishes the joint distribution of { G( z ) , z } from that of { x , E( x ) } .
Revised: To train the generator and the encoder simultaneously, they deﬁne a new objective function for the discriminator, which distinguishes the joint distribution of { G( z ) , z } from that of { x , E( x ) } .

Original: Note that E and G represent the mapping functions deﬁned by the encoder and the generator, respectively.
Revised: Note that E and G represent the mapping functions deﬁned by the encoder and the generator, respectively.

Original: Although their models can reconstruct the original image from the estimated latent vector, the visual quality of the generation is generally worse than the unidirectional GANs.
Revised: Although their models can reconstruct the original image from the estimated latent vector, the visual quality of the generation is generally worse than the unidirectional GANs.

Revised: It is because of the convergence issue as pointed by (Li et al., 2017).

Original: It is because the imperfect generator during training misleads the training of the encoder and vice versa.
Revised: We conjecture that this is because the imperfect generator during training misleads the training of the encoder and vice versa.

Original: Because of both the degraded generator and the inaccurate inference, the reconstruction quality is also poor; it is not faithful to preserving the characteristics of the original image.
Revised: Because of both the degraded generator and the inaccurate inference, the reconstruction quality is also poor; it is not faithful to preserving the characteristics of the original image.

Original: MDGAN (Che et al., 2017) and VEEGAN (Srivastava et al., 2017)also investigate the inference mapping using an encoder. Unlike ALI and BiGAN, they introduce an additional constraint, the reconstruction loss, which enforces the reconstructed images from the estimated latent vector  to be identical to the original image.
Revised: To overcome this practical convergence issue in joint distribution matching, VEEGAN (Srivastava et al., 2017) and ALICE (Li et al., 2017) introduce an additional constraint, namely reconstruction penalty, which enforces the reconstructed image (or latent) from the estimated latent vector (or image) to be identical to the original image (or latent).

Original: However, as discussed in VAEGAN, combining the reconstruction loss and adversarial loss is not trivial and frequently introduces training instability. To address this issue of training instability, MDGAN uses two discriminators and trains the generator alternatively.
Revised: Speciﬁcally, VEEGAN utilizes the inference mapping to solve mode collapse problem.

Original: That is, the ﬁrst discriminator distinguishes x from { G ( E ) , z } in order to enforce the reconstruction loss for the generator training.
Revised: To mitigate the quality degradation due to the convergence issue, they minimize the cross-entropy between P z and E ( x ), deﬁned as the reconstruction penalty in the latent space.

Original: Next, the second discriminator distinguishes { G (E) , z } from { G( z ) } to impose the adversarial loss for the generator training (i.e., the generator produces realistic images). VEEGAN adopts the idea of reconstruction loss in the latent space analogous to VAEGAN.
Revised: The idea of reconstruction penalty in the latent space is also introduced in infoGAN (Chen et al., 2016), but they do not aim to build the inference mapping. ALICE utilizes a conditional entropy, deﬁned as the cycle consistency (Zhu et al., 2017), to improve the learning instability of joint distribution matching. Although both methods improve the performance of joint distribution matching, they still su ﬀ er from the discrepancy between theoretical optimum and practical convergence (Li et al., 2017).

Original: As a result, VEEGAN subsides image blurs both in  the reconstructed and generated images compared to MDGAN.
Revised: This results in either slight blurriness in image generation or inaccurate inference mapping. Again, to bypass the convergence issue, we completely separate the training inference mapping updates to both generator updates and discriminator updates.

Original:  P roposed algorithm
Revised:  P roposed algorithm

Original: We propose a simple yet powerful algorithm to induce inference mapping based on the existing unidirectional GANs.
Revised: We propose a simple yet powerful algorithm to induce inference mapping based on the existing unidirectional GANs.

Original: We focus on increasing the accuracy and e ﬃ ciency of the inference mapping unlike recent studies for improving the GAN performance.
Revised: We focus on increasing the accuracy and e ﬃ ciency of the inference mapping unlike recent studies for improving the GAN performance.

Original: Utilizing the proposed inference mapping, we develop a new metric for evaluating the GAN performance, accounting both the image quality and diversity, by measuring thereconstruction errors of test dataset.
Revised: Utilizing the proposed inference mapping, we develop a new metric for evaluating the GAN performance, accounting both the image quality and diversity, by measuring the

Original: To build the inference mapping, we introduce the connection network.
Revised: To build the inference mapping, we introduce the connection network.

Original: The connection network transfers the feature extracted from discriminator (i.e., the discriminative feature) to the corresponding latent vector.
Revised: The connection network transfers the feature extracted from discriminator (i.e., the discriminative feature) to the corresponding latent vector.

Original: Although the discriminator is typically abandoned after GAN training, we claim that the discriminator is useful for inference mapping, because it provides rich information for establishing the generation mapping.
Revised: Although the discriminator is typically abandoned after GAN training, we claim that the discriminator is useful for inference mapping, because it provides rich information for establishing the generation mapping.

Original: Particularly, we focus on powerful features learned to represent both real and fake data and then reuse the discriminator as the feature extractor of our connection network.
Revised: Particularly, we focus on powerful features learned to represent both real and fake data and then reuse the discriminator as the feature extractor of our connection network.

Original: We insist that the discriminator as feature extractor should be more powerful representation than other encoder based feature extractors.
Revised: We insist that the discriminator as feature extractor should be more powerful representation than other encoder based feature extractors.

Original: It is because the discriminator observes both real and fake images while existing encoder based inference mapping observes either real or fake images but not both.
Revised: It is because the discriminator observes both real and fake images while existing encoder based inference mapping observes either real or fake images but not both.

Original: Furthermore, since our problem is deﬁned as the low-to-low dimensional data translation, we e ﬀ ectively gain the computational e ﬃ ciency.
Revised: Furthermore, since our problem is deﬁned as the low-to-low dimensional data translation, we e ﬀ ectively gain the computational e ﬃ ciency.

Original: As a result, our connection network is e ﬀ ective in terms of improving accuracy and e ﬃ ciency of inference mapping.
Revised: As a result, our connection network is e ﬀ ective in terms of improving accuracy and e ﬃ ciency of inference mapping.

Original: The objective for learning the connection network is represented as follows:
Revised: The objective for learning the connection network is represented as follows:

Original: [Equation]
Revised: [Equation]

Original: where CN is the connection network, D f indicates the discriminative feature vector extracted from the last layer of the discriminator.
Revised: where CN is the connection network, D f indicates the discriminative feature vector extracted from the last layer of the discriminator.

Original: Note that the generator and discriminator are ﬁxed while the connection network is updated.
Revised: Note that the generator and discriminator are ﬁxed while the connection network is updated.

Original: 3.1 C onnection network
Revised: 3.1 C onnection network

Original: Our algorithm is inspired by the report from the previous study that the GAN discriminator learns the hierarchy of features so to distinguish the real and the fake images (Radford et al., 2016).
Revised: Our algorithm is inspired by the report from the previous study that the GAN discriminator learns the hierarchy of features so to distinguish the real and the fake images (Radford et al., 2016).

Original: Based on this report, we claim that the well-trained discriminator can serve as a well-trained feature extractor for deriving the inference mapping.
Revised: Based on this report, we claim that the well-trained discriminator can serve as a well-trained feature extractor for deriving the inference mapping.

Original: In theory, GAN training is terminated when it reaches to Nash equilibrium (i.e., the generator produces realistic samples such that the discriminator no longer distinguishes the real and fake images).
Revised: In theory, GAN training is terminated when it reaches to Nash equilibrium (i.e., the generator produces realistic samples such that the discriminator no longer distinguishes the real and fake images).

Original: One might doubt that the discriminator at Nash equilibrium is no longer optimal because it cannot distinguish the fake and real images, thus not an optimal feature extractor.
Revised: One might doubt that the discriminator at Nash equilibrium is no longer optimal because it cannot distinguish the fake and real images, thus not an optimal feature extractor.

Original: If the Nash equilibrium is caused by the degradation of the discriminator power, the above statement is a reasonable argument.
Revised: If the Nash equilibrium is caused by the degradation of the discriminator power, the above statement is a reasonable argument.

Original: However, the Nash equilibrium of GAN training is not the result of weak discriminator but the result of a strong generator that produces the realistic sample.
Revised: However, the Nash equilibrium of GAN training is not the result of weak discriminator but the result of a strong generator that produces the realistic sample.

Original: Moreover, the discriminator at the Nash equilibrium is the one that leads such a strong generator.
Revised: Moreover, the discriminator at the Nash equilibrium is the one that leads such a strong generator.

Original: Hence, its discriminative features are su ﬃ ciently representable for both real and fake data.
Revised: Hence, its discriminative features are su ﬃ ciently representable for both real and fake data.

Original: Based on this analysis, we propose the connection network that learns the mapping from the discriminative feature from the discriminator to z ∼ P z .
Revised: Based on this analysis, we propose the connection network that learns the mapping from the discriminative feature from the discriminator to z ∼ P z .

Original: The generated image from z is projected onto the discriminative feature space, and then this feature vector maps to the original z using the connection network.
Revised: The generated image from z is projected onto the discriminative feature space, and then this feature vector maps to the original z using the connection network.

Original: It is important to understand that the correspondences between z and the discriminative features are automatically determined for any random variable z once training both the generator and the discriminator ends.
Revised: It is important to understand that the correspondences between z and the discriminative features are automatically determined for any random variable z once training both the generator and the discriminator ends.

Original: Because we can draw the inﬁnite number of samples from the distribution P z , the training data (i.e., a set of z and its discriminative feature vector pair) are also unlimited.
Revised: Because we can draw the inﬁnite number of samples from the distribution P z , the training data (i.e., a set of z and its discriminative feature vector pair) are also unlimited.

Original: That means, the amount of our training samples approaches to the inﬁnity, thus our training dataset can be regarded as a superset of the real dataset.
Revised: That means, the amount of our training samples approaches to the inﬁnity, thus our training dataset can be regarded as a superset of the real dataset.

Original: Owing to this attractive nature of our model, although the connection network is trained only with generated samples, we empirically observe that the inference mapping is successfully established from the real data to its latent vectors.
Revised: Owing to this attractive nature of our model, although the connection network is trained only with generated samples, we empirically observe that the inference mapping is successfully established from the real data to its latent vectors.

Original: Summarizing, as an e ﬀ ective solution for inference mapping, our connection network is superior to existing methods in terms of accuracy and computational e ﬃ ciency.
Revised: Summarizing, as an e ﬀ ective solution for inference mapping, our connection network is superior to existing methods in terms of accuracy and computational e ﬃ ciency.

Original: We believe that the accuracy is improved because the powerful feature representation from the discriminator facilitates to learn the inference mapping.
Revised: We believe that the accuracy is improved because the powerful feature representation from the discriminator facilitates to learn the inference mapping.

Original: Moreover, by reusing the features from discriminator, our problem becomes to induce the mapping from the low to the low dimensional representations.
Revised: Moreover, by reusing the features from discriminator, our problem becomes to induce the mapping from the low to the low dimensional representations.

Original: In this way, we signiﬁcantly reduce the model parameters as well as computational complexity during training.
Revised: In this way, we signiﬁcantly reduce the model parameters as well as computational complexity during training.

Original: 3.2 M etric for evaluating GAN performance
Revised: 3.2 M etric for evaluating GAN performance

Original: Suppose that our method performs an ideal inference mapping, then the reconstruction accuracy is directly related to the generator performance.
Revised: Suppose that our method performs an ideal inference mapping, then the reconstruction accuracy is directly related to the generator performance.

Original: Meanwhile, the inference performance by the connection network is governed by the e ﬀ ectiveness of discriminator as the feature extractor.
Revised: Meanwhile, the inference performance by the connection network is governed by the e ﬀ ectiveness of discriminator as the feature extractor.

Original: From this tendency, we conclude that the reconstruction performance is an important indicator of the performance of the generator and the discriminator.
Revised: From this tendency, we conclude that the reconstruction performance is an important indicator of the performance of the generator and the discriminator.

Original: Therefore, we propose a new metric that aims to evaluate whether the data distribution is accurately modeled by measuring the reconstruction errors of unseen test samples.
Revised: Therefore, we propose a new metric that aims to evaluate whether the data distribution is accurately modeled by measuring the reconstruction errors of unseen test samples.

Original: The reconstruction error is increased because 1) the generator is not strong enough to produce realistic samples (i.e., image quality), or 2) the generator does not cover all modes of data distribution (i.e., image diversity).
Revised: The reconstruction error is increased because 1) the generator is not strong enough to produce realistic samples (i.e., image quality), or 2) the generator does not cover all modes of data distribution (i.e., image diversity).

Original: For either case, we can consider that those samples are not correctly modeled by that GANs.
Revised: For either case, we can consider that those samples are not correctly modeled by that GANs.

Original: Consequently, our proposed metric is an e ﬀ ective tool for evaluating both the image quality and diversity simultaneously.
Revised: Consequently, our proposed metric is an e ﬀ ective tool for evaluating both the image quality and diversity simultaneously.

Original: To quantitatively evaluate the reconstruction performance, we employ two metrics.
Revised: To quantitatively evaluate the reconstruction performance, we employ two metrics.

Original: Given the original data and its reconstructed one, the ﬁrst is the peak signal-to-noise ratio (PSNR) that measures their average of pixel di ﬀ erence, and the other is the structural similarity index (SSIM) that measures on their structural di ﬀ erence.
Revised: Given the original data and its reconstructed one, the ﬁrst is the peak signal-to-noise ratio (PSNR) that measures their average of pixel di ﬀ erence, and the other is the structural similarity index (SSIM) that measures on their structural di ﬀ erence.

Original: In this paper, we emphasize that our metric is e ﬀ ective in addressing the limitations of the inception score and MS-SSIM, which are the most widely used metrics for evaluating GAN performance.
Revised: In this paper, we emphasize that our metric is e ﬀ ective in addressing the limitations of the inception score and MS-SSIM, which are the most widely used metrics for evaluating GAN performance.

Original: The inception score evaluates the quality of generated images using a pre-trained classiﬁer.
Revised: The inception score evaluates the quality of generated images using a pre-trained classiﬁer.

Original: The critical, well-known issue on inception score is that poor quality images do not necessarily map to low inception scores as discussed in (Barratt & Sharma, 2018); Salimans et al.
Revised: The critical, well-known issue on inception score is that poor quality images do not necessarily map to low inception scores as discussed in (Barratt & Sharma, 2018); Salimans et al.

Original: (2016) shows that adversarial examples can greatly increase the inception score although they have poor quality.
Revised: (2016) shows that adversarial examples can greatly increase the inception score although they have poor quality.

Original: The
Revised: The

Original: MS-SSIM is the most popular metric for measuring the diversity of generated images.
Revised: MS-SSIM is the most popular metric for measuring the diversity of generated images.

Original: However, it is not e ﬀ ective if the target dataset consists of very diverse images (Fedus et al., 2018) such as
Revised: However, it is not e ﬀ ective if the target dataset consists of very diverse images (Fedus et al., 2018) such as

Original: CIFAR10 or ImageNet.
Revised: CIFAR10 or ImageNet.

Original: More importantly, MS-SSIM can be manipulated by simply generating random noises because it only accounts the patch di ﬀ erence among generated images.
Revised: More importantly, MS-SSIM can be manipulated by simply generating random noises because it only accounts the patch di ﬀ erence among generated images.

Original: Summarizing, both inception score and MS-SSIM are not robust because there are alternative ways to attack those metrics, generating either adversarial examples or random patches.
Revised: Summarizing, both inception score and MS-SSIM are not robust because there are alternative ways to attack those metrics, generating either adversarial examples or random patches.

Original: Unlike existing metrics, the proposed metric rigorously measures the image quality regardless of datasets; a poor (or high) quality image always maps to a low (or high) score on any dataset.
Revised: Unlike existing metrics, the proposed metric rigorously measures the image quality regardless of datasets; a poor (or high) quality image always maps to a low (or high) score on any dataset.

Original: When adapting PSNR as the di ﬀ erence measure, the di ﬀ erence between the original image and that with a slight translation can be large although they are nearly identical in terms of image context.
Revised: When adapting PSNR as the di ﬀ erence measure, the di ﬀ erence between the original image and that with a slight translation can be large although they are nearly identical in terms of image context.

Original: Fortunately, this issue can be resolved with SSIM.
Revised: Fortunately, this issue can be resolved with SSIM.

Original: Moreover, our metric is robust against attacks because we explicitly compare the reconstruction with the ground truth.
Revised: Moreover, our metric is robust against attacks because we explicitly compare the reconstruction with the ground truth.

Original: Note that various issues arose in both inception score and MS-SSIM are often caused by the absence of ground truth.
Revised: Note that various issues arose in both inception score and MS-SSIM are often caused by the absence of ground truth.

Original: Another attractive property of the proposed metric is that it is applicable to any dataset, and our score is intuitive to interpret the performance.
Revised: Another attractive property of the proposed metric is that it is applicable to any dataset, and our score is intuitive to interpret the performance.

Original: Note that not all dataset can be evaluated by both metrics.
Revised: Note that not all dataset can be evaluated by both metrics.

Original: For example, CelebA cannot be evaluated by inception score and CIFAR10 cannot be evaluated with
Revised: For example, CelebA cannot be evaluated by inception score and CIFAR10 cannot be evaluated with

Original: MS-SSIM (Odena et al., 2017).
Revised: MS-SSIM (Odena et al., 2017).

Original: Also, inception score and MS-SSIM should be compared with its ideal performance, the score of the real dataset, to understand its level of achievement.
Revised: Also, inception score and MS-SSIM should be compared with its ideal performance, the score of the real dataset, to understand its level of achievement.

Original: That is, it is di ﬃ cult to intuitively interpret the quality from absolute numbers.
Revised: That is, it is di ﬃ cult to intuitively interpret the quality from absolute numbers.

Original: On the other hand, the proposed metric has an absolute criterion; SSIM
Revised: On the other hand, the proposed metric has an absolute criterion; SSIM

Original: = 1, PSNR → inf dB for ideal performance while SSIM = 0,
Revised: = 1, PSNR → inf dB for ideal performance while SSIM = 0,

Original: PSNR → 0 dB for the worst performance.
Revised: PSNR → 0 dB for the worst performance.

Original: As mentioned earlier, our metric evaluates both the image quality and diversity with a single score while existing metrics focus on either of them.
Revised: As mentioned earlier, our metric evaluates both the image quality and diversity with a single score while existing metrics focus on either of them.

Original: Because the proposed metric utilizing image reconstruction penalizes either poor quality or poor diversity, it reports high scores only if both quality and diversity are satisﬁed.
Revised: Because the proposed metric utilizing image reconstruction penalizes either poor quality or poor diversity, it reports high scores only if both quality and diversity are satisﬁed.

Original: It is clear that PSNR and SSIM measure the quality of a single sample, thus the average of many samples generalizes the image quality.
Revised: It is clear that PSNR and SSIM measure the quality of a single sample, thus the average of many samples generalizes the image quality.

Original: Unlike the image quality, it might be less intuitive to evaluate the image diversity by image reconstruction.
Revised: Unlike the image quality, it might be less intuitive to evaluate the image diversity by image reconstruction.

Original: We claim that the image diversity can be evaluated by the average PSNR or SSIM of many samples if those samples cover all modes of the data distribution; it is achieved by constructing the test dataset by uniformly sampling the data distribution.
Revised: We claim that the image diversity can be evaluated by the average PSNR or SSIM of many samples if those samples cover all modes of the data distribution; it is achieved by constructing the test dataset by uniformly sampling the data distribution.

Original: If the baseline GANs cannot cover various modes of data distribution, the image reconstruction is generally inaccurate, leads to the low score.
Revised: If the baseline GANs cannot cover various modes of data distribution, the image reconstruction is generally inaccurate, leads to the low score.

Original:  E xperimental evaluation
Revised:  E xperimental evaluation

Original: In this section, we evaluate how accurately the proposed algorithm reconstructs the original images qualitatively and quantitatively compared with a naive encoder based inference mapping, VAE and
Revised: In this section, we evaluate how accurately the proposed algorithm reconstructs the original images qualitatively and quantitatively compared with a naive encoder based inference mapping, VAE and

Original: ALI / BiGAN.
Revised: ALI / BiGAN.

Original: For the comparison, we use ﬁve di ﬀ erent unidirectional GANs as baseline networks and apply our connection network for inference mapping.
Revised: For the comparison, we use ﬁve di ﬀ erent unidirectional GANs as baseline networks and apply our connection network for inference mapping.

Original: Those of baseline networks are DCGAN (Radford et al., 2016), LSGAN (Mao et al., 2017), DFM (Warde-Farley & Bengio, 2017) RFGAN
Revised: Those of baseline networks are DCGAN (Radford et al., 2016), LSGAN (Mao et al., 2017), DFM (Warde-Farley & Bengio, 2017), RFGAN

Original: Bang & Shim (2018) and WGAN-GP (Gulrajani et al., 2017).
Revised: Bang & Shim (2018) and WGAN-GP (Gulrajani et al., 2017).

Original: We intend to choose those ﬁve baseline networks because all ﬁve models are signiﬁcantly di ﬀ erent in terms of loss functions or network architectures.
Revised: We intend to choose those ﬁve baseline networks because all ﬁve models are signiﬁcantly di ﬀ erent in terms of loss functions or network architectures.

Original: For example, DCGAN, LSGAN, and WGAN-GP exploit di ﬀ erent metrics while
Revised: For example, DCGAN, LSGAN, and WGAN-GP exploit di ﬀ erent metrics while

Original: [Figure]

Original: DFM adds a denoising autoencoder to the discriminator for the robustness and RFGAN implicitly regularizes the discriminator using representative features.
Revised: DFM adds a denoising autoencoder to the discriminator for the robustness and RFGAN implicitly regularizes the discriminator using representative features.

Original: Evaluating with a variety of unidirectional
Revised: Evaluating with a variety of unidirectional

Original: GANs, we aim to show 1) the extendibility of the proposed algorithm and 2) the e ﬀ ectiveness of the proposed metric using the connection network.
Revised: GANs, we aim to show 1) the extendibility of the proposed algorithm and 2) the e ﬀ ectiveness of the proposed metric using the connection network.

Original: For the fair evaluation, the architecture of all baseline networks (e.g., the number of layers, ﬁlter size, using batch normalization) borrows that of DCGAN.
Revised: For the fair evaluation, the architecture of all baseline networks (e.g., the number of layers, ﬁlter size, using batch normalization) borrows that of DCGAN.

Original: The connection network is composed of only two fully connected layers: 1024 full connected layer (FC) – Batch normalization (BN) – leaky rectiﬁed linear unit (Leaky ReLU) – 1024 FC – BN – Leaky
Revised: The connection network is composed of only two fully connected layers: 1024 full connected layer (FC) – Batch normalization (BN) – leaky rectiﬁed linear unit (Leaky ReLU) – 1024 FC – BN – Leaky

Original: ReLU – dimension of P z FC.
Revised: ReLU – dimension of P z FC.

Original: For experimental evaluation, we utilize three di ﬀ erent real datasets: Fashion MNIST (Xiao et al., 2017), CIFAR10 (Krizhevsky & Hinton, 2009), and CelebA (Liu et al., 2015), all normalized between − 1 and 1.
Revised: For experimental evaluation, we utilize three di ﬀ erent real datasets: Fashion MNIST (Xiao et al., 2017), CIFAR10 (Krizhevsky & Hinton, 2009), and CelebA (Liu et al., 2015), all normalized between − 1 and 1.

Original: Note that the input dimensionality of Fashion MNIST is (28 , 28 , 1), CIFAR10 is (32 , 32 , 3), and CelebA is (64 , 64 , 3).
Revised: Note that the input dimensionality of Fashion MNIST is (28 , 28 , 1), CIFAR10 is (32 , 32 , 3), and CelebA is (64 , 64 , 3).

Original: Evaluating the inference mapping.
Revised: Evaluating the inference mapping.

Original: First of all, we investigate the e ﬀ ectiveness of the discriminator’s feature in the  inference mapping by comparing ours with the naïve encoder mapping. For that, we choose DCGAN as the baseline GANs.
Revised: We compare the proposed method with 1) naïve encoder encoder mapping, 2) iGAN (naïve encoder encoder followed by optimization in the image domain) and 3) hybrid method using the proposed method (discriminator with CN followed by optimization). Furthermore, to investigate the capability of the discriminator as the feature extractor, we directly compare our inference mapping (discriminator with CN network) with the VGG16 model (Simonyan & Zisserman, 2015) based inference mapping (pre-trained VGG16 with CN network).

Original: For our inference mapping, the connection network is added to the pretrained DCGAN.
Revised: Note that the VGG16 model pre-trained with ImageNet is known as the good feature extractor that characterizes perceptually meaningful representation. Fig 1 provides the reconstructed images from each method at three di ﬀ erent iterations and the quantitative evaluation.

Original: To establish the naïve encoder mapping (i.e., from the fake image to z ), unlike existing bidirectional GANs, we train the encoder with the ﬁxed, pretrained DCGAN similar to our algorithm. The encoder structure is identical to the discriminator except for the last layer, which dimension equals to the dimension ofz .
Revised: Our algorithm successfully synthesizes the attributes in various faces, unlike the naïve encoder. As reported in iGAN, we conﬁrm that adopting the non-convex optimization for inference mapping signiﬁcantly enhances the quantitative score (i.e., SSIM and PSNR). It is because the non-convex optimization directly minimizes the pixel-wise di ﬀ erence between test images and reconstructed images; the goal of the non-convex optimization is nearly equivalent to the goal of

Original: Fig 1 demonstrates the reconstructed images from ours and naïve encoder at three di ﬀ erent iterations.
Revised: PSNR. Hence, the hybrid method improves the PSNR of any baseline encoder mapping.

Original: For each iteration, the ﬁrst and fourth columns indicate the input images, the second and ﬁfth columns are the reconstructed image using our connection network, and the third and sixth columns are from the naïve encoder.
Revised: When we replace the naïv encoder with the proposed inference mapping, its quantitative results are better than iGAN. It is because our inference mapping predicts more accurate initial latent vector. However, these quantitative results do not exactly match with qualitative results. The quantitative results demonstrate that hybrid inference mapping is the most e ﬀ ective among all others.

Revised: Meanwhile, the qualitative results from the hybrid methods are generally blur or have missing important components (e.g., eye glasses, mustache, gender, wrinkles, detailed hair lines, etc.).

Revised: Because the hybrid inference mapping optimizes the inference mapping in the image domain (i.e., minimizing the pixel-wise di ﬀ erence), the inference network ﬁnally chooses the latent vector corresponding to an average-like image.

Revised: Note that there exists average-like faces among many possible faces.

Revised: We conjecture that, although the generator can produce sharp images, the hybrid inference mapping strategically selects average-like faces to reduce its loss function.

Original: In all cases, our model provides better results than the naïve encoder qualitatively and quantitatively, in terms of visual ﬁdelity and  the reconstruction accuracy.
Revised: Meanwhile, our method (also VGG16 based inference mapping) optimizes the inference mapping in the latent domain. Thus, our inference results are sharp and better preserve semantically important attributes. From examples shown in Fig 1 and Appendix Fig 6, pixel-wise loss based methods (i.e., iGAN and Hybrid + ours) fail to capture glasses, but latent vector loss based methods (i.e., ours and VGG16) reproduce the glasses.

Revised: The additional qualitative and quantitative results are presented in Appendix B.

Revised: By replacing the discriminator as feature extractor by the pre-trained VGG16 network, we observe that its inference results are also as sharp and realistic as our results.

Revised: However, considering the semantic similarity between the original and reconstructed image, our inference mapping can restore unique attributes (e.g., mustache, race, age, etc.)

Revised: better than the VGG16 based inference mapping.

Revised: Moreover, utilizing the pre-trained VGG16 require additional memory overhead while our method does not.

Revised: In terms of network capacity, VGG16 has the much deeper network than the discriminator.

Revised: Thus, we conclude that the proposed inference mapping is more e ﬃ cient than the VGG16 based inference mapping.

Original: From this experiment, we conﬁrm that our idea of reusing discriminator’s features ismore e ﬀ ective than building the new encoder for feature extractor.
Revised: From these results, we conﬁrm that recycling discriminator as a feature extractor is e ﬀ ective for improving inference accuracy and reducing the computational complexity. Ours Naïve encoder iGAN Hybrid + Ours VGG

Original:  As mentioned earlier, VAE and ALI / BiGAN are representative generative models that allow inference mapping.
Revised: SSIM 0. 0.5199 PSNR(dB) 16. As mentioned earlier, VAE and ALI / BiGAN are representative generative models that allow inference mapping.

Original: In the following experiment, we compare our models with VAE and ALI / BiGAN.
Revised: In the following experiment, we compare our models with VAE and ALI / BiGAN.

Original: Figvisualizes reconstructed images from VAE, ALI / BiGAN, and our models applied to ﬁve di ﬀ erent baseline GANs, when the input images are at odd columns and their reconstructed images are at next even columns.
Revised: Figvisualizes reconstructed images from VAE, ALI / BiGAN, and our models applied to ﬁve di ﬀ erent baseline GANs, when the input images are at odd columns and their reconstructed images are at next even columns.

Original: The corresponding reconstruction accuracy in terms of PSNR and SSIM is summarized in Table 1.
Revised: The corresponding reconstruction accuracy in terms of PSNR and SSIM is summarized in Table 1.

Original: The generated images from VAE are blurry and lose the detail structures because it is optimized with a pixel reconstruction loss.
Revised: The generated images from VAE are blurry and lose the detail structures because it is optimized with a pixel reconstruction loss.

Original: Also, the less frequently appearing attributes (e.g., mustache and baldness) in the training dataset are rarely recovered in their reconstructed images.
Revised: Also, the less frequently appearing attributes (e.g., mustache and baldness) in the training dataset are rarely recovered in their reconstructed images.

Original: ALI / BiGAN generates sharper images than VAE.
Revised: ALI / BiGAN generates sharper images than VAE.

Original: However, they are not e ﬀ ective to restore the important characteristics of the input image (e.g., identity), and occasionally generate completely di ﬀ erent images from the input images.
Revised: However, they are not e ﬀ ective to restore the important characteristics of the input image (e.g., identity), and occasionally generate completely di ﬀ erent images from the input images.

Original: Also, from Table 1, we conﬁrm that the accuracy of our models in both PSNR and SSIM always outperforms that of VAE and that of ALI / BiGAN.
Revised: Also, from Table 1, we conﬁrm that the accuracy of our models in both PSNR and SSIM always outperforms that of VAE and that of ALI / BiGAN.

Original: Likewise, we observe that the reconstructed images from the variants of our models exhibit consistently better visual quality than both VAE and ALI / BiGAN.
Revised: Likewise, we observe that the reconstructed images from the variants of our models exhibit consistently better visual quality than both VAE and ALI / BiGAN.

Original: Because training our connection network focuses on the accurate inference mapping without inﬂuencing the training of the baseline GANs, the reconstructed image quality from ours is identical to that of the baseline unidirectional GANs: sharp and realistic.
Revised: Because training our connection network focuses on the accurate inference mapping without inﬂuencing the training of the baseline GANs, the reconstructed image quality from ours is identical to that of the baseline unidirectional GANs: sharp and realistic.

Original: Furthermore, our results are superior to VAE and ALI / BiGAN in that we faithfully reconstruct the input images including various facial attributes while VAE and ALI / BiGAN often fail to handle.
Revised: Furthermore, our results are superior to VAE and ALI / BiGAN in that we faithfully reconstruct the input images including various facial attributes while VAE and ALI / BiGAN often fail to handle.

Original: Based on these observations we conclude that the proposed algorithm accurately estimates the latent vector corresponding to the input image and retains the image quality, better than other competitors.
Revised: Based on these observations, we conclude that the proposed algorithm accurately estimates the latent vector corresponding to the input image and retains the image quality, better than other competitors.

Original: Experiment for image editing.
Revised: Experiment for image editing

Original: Previously, we have shown that the latent vector corresponding to the image can be accurately estimated using our inference mapping.
Revised: Previously, we have shown that the latent vector corresponding to the image can be accurately estimated using our inference mapping.

Original: One of the popular applications utilizing the inference mapping is an image editing by disentangling the attributes and then modifying them for semantic editing.
Revised: One of the popular applications utilizing the inference mapping is an image editing by disentangling the attributes and then modifying them for semantic editing.

Original: Previous study (Mikolov et al., 2013) reported that the simple arithmetic operation in the latent space of GANs produces a semantically meaningful combination in the representation space.
Revised: Previous study (Mikolov et al., 2013) reported that the simple arithmetic operation in the latent space of GANs produces a semantically meaningful combination in the representation space.

Original: Later, in DCGAN, the latent vector arithmetic is successful for facial attribute editing and produces various image e ﬀ ects for the ﬁrst time.
Revised: Later, in DCGAN, the latent vector arithmetic is successful for facial attribute editing and produces various image e ﬀ ects for the ﬁrst time.

Original: However, estimating the latent vector for the target image  is not trivial.
Revised: However, estimating the latent vector for the target image [Table] is not trivial.

Original: Even utilizing the state-of-the-art non-convex optimization, it often falls into local minima and computationally intractable for real applications.
Revised: Even utilizing the state-of-the-art non-convex optimization, it often falls into local minima and computationally intractable for real applications.

Original: The proposed method is attractive in that we achieve the accurate latent estimation in real time (i  e., forward propagation).
Revised: The proposed method is attractive in that we achieve the accurate latent estimation in real time (i [Table] e., forward propagation).

Original: That means, our algorithm can be used to edit image attributes in real time.
Revised: That means, our algorithm can be used to edit image attributes in real time.

Original: We demonstrate the results from the variants of our models as shown in Fig 3.
Revised: We demonstrate the results from the variants of our models as shown in Fig 3.

Original: From this ﬁgure, the ﬁrst to the second row are the result of adding the mean latent vector for blonde hair with the female face, and the third to fourth column are the result of adding the mean latent vector for glasses with the male face; the images are arranged in the order of test image, reconstructed images, and vector arithmetic edited image.
Revised: From this ﬁgure, the ﬁrst to the second row are the result of adding the mean latent vector for blonde hair with the female face, and the third to fourth column are the result of adding the mean latent vector for glasses with the male face; the images are arranged in the order of test image, reconstructed images [Table] and vector arithmetic edited image.

Original: In all cases, our algorithm successfully synthesizes the attributes in various faces.
Revised: In all cases, our algorithm successfully synthesizes the attributes in various faces.

Original: Experimental comparison for GAN metric.
Revised: Experimental comparison for GAN metric.

Original: As mentioned in Sec 3.2, we measure the reconstruction accuracy using PSNR and SSIM on test dataset and use it as a new quantitative metric for evaluating the GAN performance.
Revised: As mentioned in Sec 3.2, we measure the reconstruction accuracy using PSNR and SSIM on test dataset and use it as a new quantitative metric for evaluating the GAN performance.

Original: Similar to the previous experiment, VAE and ALI / BiGAN are compared with ﬁve variants of our algorithms.
Revised: Similar to the previous experiment, VAE and ALI / BiGAN are compared with ﬁve variants of our algorithms.

Original: For the fair evaluation, we use a total of 1k test images for the experiment and then report their mean and standard deviation in Table 1, both in terms of
Revised: For the fair evaluation, we use a total of 1k test images for the experiment and then report their mean and standard deviation in Table 1, both in terms of

Original: PSNR and SSIM repeated ﬁve times for each model and each dataset.
Revised: PSNR and SSIM repeated ﬁve times for each model and each dataset.

Original: We choose 1k for test samples because the reconstruction error does not change much by increasing the test samples, varying from 100 to 10k.
Revised: We choose 1k for test samples because the reconstruction error does not change much by increasing the test samples, varying from 100 to 10k.

Original: While inception score and MS-SSIM are meaningful on a speciﬁc dataset, the proposed[Table] metric is applicable for any type of dataset.
Revised: While inception score and MS-SSIM are meaningful on a speciﬁc dataset, the proposed metric is applicable for any type of dataset.

Original: Note that we evaluate various GANs using three di ﬀ erent resolution and di ﬀ erent channel dataset (i.e., Fashion MNIST, CIFAR10, and CelebA) and show the consistent results, having similar ranks across di ﬀ erent datasets.
Revised: Note that we evaluate various GANs using three di ﬀ erent resolution and di ﬀ erent channel dataset (i.e., Fashion MNIST, CIFAR10, and CelebA) and show the consistent results, having similar ranks across di ﬀ erent datasets.

Original: Comparing the existing metrics on their target dataset (i.e., inception score on CIFAR10 and MSSSIM on CelebA), we observe that the proposed metric reports similar tendency; top-2 results from ours and existing metrics signiﬁcantly overlap.
Revised: Comparing the existing metrics on their target dataset (i.e., inception score on CIFAR10 and MSSSIM on CelebA), we observe that the proposed metric reports similar tendency; top-2 results from ours and existing metrics signiﬁcantly overlap.

Original: By closely analyzing each metric, our metric di ﬀ ers from others in that we account both image quality and diversity simultaneously.
Revised: By closely analyzing each metric, our metric di ﬀ ers from others in that we account both image quality and diversity simultaneously.

Original: For example, our metric reports the worst score for ALI / BiGAN, which has relatively a good score (top-3) in MSSSIM.
Revised: For example, our metric reports the worst score for ALI / BiGAN, which has relatively a good score (top-3) in MSSSIM.

Original: This is reasonable because ALI / BiGAN utilizes the bidirectional mapping to improve the diversity but sacriﬁcing the image quality.
Revised: This is reasonable because ALI / BiGAN utilizes the bidirectional mapping to improve the diversity but sacriﬁcing the image quality.

Original: Since MS-SSIM only accounts the diversity, ALI / BiGAN achieves the meaningful performance with MS-SSIM.
Revised: Since MS-SSIM only accounts the diversity, ALI / BiGAN achieves the meaningful performance with MS-SSIM.

Original: However, our metric measures the overall performance, thereby ALI / BiGAN is penalized due to its quality degradation.
Revised: However, our metric measures the overall performance, thereby ALI / BiGAN is penalized due to its quality degradation.

Original: Because our metric explicitly provides the absolute numbers comparable with di ﬀ erent datasets, we can approximate the di ﬃ culty of di ﬀ erent datasets; most GANs achieve relatively higher performance on CelebA than others.
Revised: Because our metric explicitly provides the absolute numbers comparable with di ﬀ erent datasets, we can approximate the di ﬃ culty of di ﬀ erent datasets; most GANs achieve relatively higher performance on CelebA than others.

Original: This is because CelebA includes roughly aligned images with less diversity than other datasets.
Revised: This is because CelebA includes roughly aligned images with less diversity than other datasets.

Original:  C onclusion
Revised:  C onclusion

Original: In this study, we propose a new algorithm for establishing the accurate and e ﬃ cient inference mapping on existing unidirectional GANs, without sacriﬁcing their image quality.
Revised: In this study, we propose a new algorithm for establishing the accurate and e ﬃ cient inference mapping on existing unidirectional GANs, without sacriﬁcing their image quality.

Original: For that, we introduce the connection network that transfers the discriminative features extracted from the last layer of the discriminator to the latent vector.
Revised: For that, we introduce the connection network that transfers the discriminative features extracted from the last layer of the discriminator to the latent vector.

Original: Since the discriminator learns to distinguish between real and fake samples, its features should be representable for both real and fake images.
Revised: Since the discriminator learns to distinguish between real and fake samples, its features should be representable for both real and fake images.

Original: Hence, we expect that recycling the discriminator’s features for building inference mapping leads to more accurate inference mapping than existing methods; existing methods adopt a new feature extractor associated either fake images or real images, but not both.
Revised: Hence, we expect that recycling the discriminator’s features for building inference mapping leads to more accurate inference mapping than existing methods; existing methods adopt a new feature extractor associated either fake images or real images, but not both.

Original: Moreover, because learning the transfer from feature to latent vector is modeled by a low-to-low dimensional mapping, our algorithm is computationally e ﬃ cient for training.
Revised: Moreover, because learning the transfer from feature to latent vector is modeled by a low-to-low dimensional mapping, our algorithm is computationally e ﬃ cient for training.

Original: In this paper, we introduce two major applications using the e ﬃ cient inference mapping: 1) manipulating the image by latent vector arithmetic and 2) suggesting a new metric for evaluating GAN performance by measuring the reconstruction error using unseen real data.
Revised: In this paper, we introduce two major applications using the e ﬃ cient inference mapping: 1) manipulating the image by latent vector arithmetic and 2) suggesting a new metric for evaluating GAN performance by measuring the reconstruction error using unseen real data.

Original: Especially, we empirically show that our metric explicitly measures GAN performance and is applicable for various datasets with the di ﬀ erent resolution / channel while existing metrics are limited to speciﬁc datasets (i.e., Inception score and MS-SSIM).
Revised: Especially, we empirically show that our metric explicitly measures GAN performance and is applicable for various datasets with the di ﬀ erent resolution / channel while existing metrics are limited to speciﬁc datasets (i.e., Inception score and MS-SSIM).

Original: We expect that the proposed algorithm can serve the important basis of future studies for analyzing and evaluating GAN models.
Revised: We expect that the proposed algorithm can serve the important basis of future studies for analyzing and evaluating GAN models.

Original: A ppendix 
Revised: A ppendix A. Application

Original: Toward the advanced image editing application, we develop another a novel framework for simultaneously generating and manipulating the face images with desired attributes.
Revised: Toward the advanced image editing application, we develop another a novel framework for simultaneously generating and manipulating the face images with desired attributes.

Original: This work utilizes the connection network proposed in this paper and is under review for the publication.
Revised: This work utilizes the connection network proposed in this paper and is under review for the publication.

Original: The goal of this work is to develop a single uniﬁed model that can simultaneously create and edit high-quality face images with desired attributes.
Revised: The goal of this work is to develop a single uniﬁed model that can simultaneously create and edit high-quality face images with desired attributes.

Original: The proposed framework decomposes the image into the latent and attribute vector in low dimensional representation through the connection network, and then utilizes the GAN framework for mapping the low dimensional representation to the image.
Revised: The proposed framework decomposes the image into the latent and attribute vector in low dimensional representation through the connection network, and then utilizes the GAN framework for mapping the low dimensional representation to the image.

Original: Fig.
Revised: Fig.

Original: 4 shows the architecture of this model.
Revised: 4 shows the architecture of this model.

Original: In this way, this new model can handle both the generation and editing problem by learning the generator.
Revised: In this way, this new model can handle both the generation and editing problem by learning the generator.

Original: Fig.
Revised: Fig.

Original: 5 shows the results of attribute editing from the VAE / GAN (Larsen et al., 2016), the modiﬁed cGAN (Mirza & Osindero, 2014), IcGAN (Perarnau et al., 2016), and the new model using the connection network.
Revised: 5 shows the results of attribute editing from the VAE / GAN (Larsen et al., 2016), the modiﬁed cGAN (Mirza & Osindero, 2014), IcGAN (Perarnau et al., 2016), and the new model using the connection network.

Original: This new model achieves the competitive performance with the state-of-the-art attribute editing technique in terms of reconstruction and attributes editing quality. 
Revised: This new model achieves the competitive performance with the state-of-the-art attribute editing technique in terms of reconstruction and attributes editing quality. [Figure] . Qualitative

Revised: [Figure]

Revised: In the section 4, we quantitatively and qualitatively evaluate the proposed inference mapping compared with three di ﬀ erent inference methods and conﬁrm recycling discriminator is suitable for the feature extractor.

Revised: Furthermore, we analyze the reason for the gap between quantitative results and qualitative results.

Revised: Although our quantitative results are somewhat worse than those of hybrid methods, we believe that our inference mapping (i.e., adopting the inference mapping through optimizing on the latent space) is still the better choice.

Revised: It is because our inference mapping preserves the desirable properties of GANs, which generate sharp images with various semantic attributes.

Revised: We believe that reproducing the semantic attributes could be interpreted as how well GANs understand the images.

Revised: In this regard, because the hybrid method tends to choose average-like images even omitting semantic attributes, it is not appropriate to adopt the hybrid method for building the GAN evaluation metric.

Revised: Also, by recycling the discriminator, the quality of discriminator directly a ﬀ ects the accuracy of inference mapping.

Revised: Thus, our method is suitable for evaluating the entire GAN framework; we utilize both the generator and the discriminator for evaluating the GAN model.