Paper:
Generative adversarial networks (GANs) have achieved outstanding success in generating the high-quality data. Focusing on the generation process, existing GANs learn a unidirectional mapping from the latent vector to the data. Later, various studies point out that the latent space of GANs is semantically meaningful and can be utilized for advanced data analysis and manipulation. In order to analyze the real data in the latent space of GANs, it is necessary to investigate the inverse generation mapping from the data to the latent vector. To tackle this problem, the bidirectional generative models introduce an encoder to establish the inverse path of the generation process. Unfortunately, their inference mapping does not accurately predict the latent vector from the data because the imperfect generator rather interferes the encoder training. In this paper, we propose an effective algorithm to accurately infer the latent vector based on existing unidirectional GANs. It is important to note that we focus on increasing the accuracy and efficiency of the inference mapping but not influencing the GAN performance toward image quality or diversity. Instead, utilizing the proposed inference mapping algorithm, we suggest a new metric for evaluating the GAN models by measuring the reconstruction error of unseen real data. The experimental analysis demonstrates that the proposed algorithm achieves more accurate inference mapping than existing methods and provides the robust metric for evaluating GAN performance.

Review:
Paper Summary: 
This paper proposes to reconstruct the generated images to the their corresponding latent code. As claimed, the goal is to improve the accuracy and efficiency of inference mapping better than other inference mapping techniques, while maintaining their generation quality.
 Instead of using an independent encoder, the authors propose to share the encoder parameters with the discriminator: a Connection Network (CN) is built on top of the features extracted by the discriminator. The weight-sharing machisme shows better performance in Figure 1.
The proposed method has two benefits: : a) manipulating the image by disentangling the latent space and b) suggesting a new metric for assessing the GAN model by measuring reconstruction errors of real data.
General Comments:
In term of algorithm, the paper essentially adds the conscontruction term (CN) to the standard GAN loss, and partially shares the weights of the “encoder” and discriminator. However, it is almost identical to the existing works, which are NOT cited, and the connections are not discussed.
Connection to InfoGAN: To relate the generated images to the latent code,  the proposed method employs the reconstruction loss, InfoGAN employs the mutual information. Note that reconstruction loss = negative log likelihood, and effectively is equivalent to Mutual Information and Conditional Entropy in the case. Please see the discussion in Lemma 3 and Appendix A of [3] for detailed discussion.  Further, InfoGAN has proposed to to sharing weights of the encoder and discriminator, exactly the same with this submission. The claimed advantage is to disentangle the latent space. It is not surprise at all, once the authors see the connection to InfoGAN, which was originally proposed to disentangle the latent codes.
Connection to CycleGAN: CycleGAN consists of four losses: two reconstruction losses and two standard GAN losses. As shown in Section 4 of [3] “Connecting ALI and CycleGAN”, one reconstruction loss and  one standard GAN loss is sufficient to achieve CycleGAN’s objective, the other two losses would only help to accelerate. In another word, the proposed method is exactly half of the CycleGAN losses.
The author mention in Abstract that “the bidirectional generative models introduce an encoder to establish the inverse path of the generation process. Unfortunately, their inference mapping does not accurately predict the latent vector from the data because the imperfect generator rather interferes the encoder training.” This is the non-identifiable issue of ALI/BiGAN discovered in [3]. Please clarify. 
The proposed method should compare with [1] and [2] in great detail, to demonstrate its own advantages. Given the missing literature, the current experimental comparisons seem not that meaningful, because the baseline methods are not really the competitors. 
One interesting contribution of the submission is to consider the reconstruction errors to measure the quality of GANs. To my best knowledge, it is original. 
References:
[1] InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets, NIPS 2016
[2] Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks, ICCV 2017
[3] ALICE: Towards Understanding Adversarial Learning for Joint Distribution Matching, NIPS 2017