Paper:
Disentangling factors of variation has always been a challenging problem in representation learning. Existing algorithms suffer from many limitations, such as unpredictable disentangling factors, bad quality of generated images from encodings, lack of identity information, etc. In this paper, we proposed a supervised algorithm called DNA-GAN trying to disentangle different attributes of images. The latent representations of images are DNA-like, in which each individual piece represents an independent factor of variation. By annihilating the recessive piece and swapping a certain piece of two latent representations, we obtain another two different representations which could be decoded into images. In order to obtain realistic images and also disentangled representations, we introduced the discriminator for adversarial training. Experiments on Multi-PIE and CelebA datasets demonstrate the effectiveness of our method and the advantage of overcoming limitations existing in other methods.

Review:
Summary:
This paper investigated the problem of attribute-conditioned image generation using generative adversarial networks. More specifically, the paper proposed to generate images from attribute and latent code as high-level representation. To learn the mapping from image to high-level representations, an auxiliary encoder was introduced. The model was trained using a combination of reconstruction (auto-encoding) and adversarial loss. To further encourage effective disentangling (against trivial solution), an annihilating operation was proposed together with the proposed training pipeline. Experimental evaluations were conducted on standard face image databases such as Multi-PIE and CelebA.
== Novelty and Significance ==
Multi-attribute image generation is an interesting task but has been explored to some extent. The integration of generative adversarial networks with auto-encoding loss is not really a novel contribution.
-- Autoencoding beyond pixels using a learned similarity metric. Larsen et al., In ICML 2016.
== Technical Quality == 
First, it is not clear how was the proposed annihilating operation used in the experiments (there is no explanation in the experimental section). Based on my understanding, additional loss was added to encourage effective disentangling (prevent trivial solution). I would appreciate the authors to elaborate this a bit.
Second, the iterative training (section 3.4) is not a novel contribution since it was explored in the literature before (e.g., Inverse Graphics network). The proof developed in the paper provides some theoretical analysis but cannot be considered as a significant contribution.
Third, it seems that the proposed multi-attribute generation pipeline works for binary attribute only. However, such assumption limits the generality of the work. Since the title is quite general, I would assume to see the results (1) on datasets with real-valued attributes, mixture attributes or even relative attributes and (2) not specific to face images.
-- Learning to generate chairs with convolutional neural networks. Dosovitskiy et al., In CVPR 2015.
-- Deep Convolutional Inverse Graphics Network. Kulkarni et al., In NIPS 2015.
-- Attribute2Image: Conditional Image Generation from Visual Attributes. Yan et al., In ECCV 2016.
-- InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets. Chen et al., In NIPS 2016.
Additionally, considering the generation quality, the CelebA samples in the paper are not the state-of-the-art. I suspect the proposed method only works in a more constrained setting (such as Multi-PIE where the images are all well aligned).
Overall, I feel that the submitted version is not ready for publication in the current form.