Paper:
We propose a new architecture and training methodology for generative adversarial networks. Current approaches attempt to learn the transformation from a noise sample to a generated data sample in one shot. Our proposed generator architecture, called ChainGAN, uses a two-step process. It first attempts to transform a noise vector into a crude sample, similar to a traditional generator. Next, a chain of networks, called editors, attempt to sequentially enhance this sample. We train each of these units independently, instead of with end-to-end backpropagation on the entire chain. Our model is robust, efficient, and flexible as we can apply it to various network architectures. We provide rationale for our choices and experimentally evaluate our model, achieving competitive results on several datasets.

Review:
Summary:
The authors present a straightforward method to improve generative quality of GANs that can allow for fewer parameters by separating the task into a basic-generation followed by a chain of multiple edits to the base generation with different editor networks. The fact that separating the task allows for smaller networks is key to the reduction in parameters.  Each editor is trained separately in alternance, with its associated critic. Authors test their approach mostly on CIFAR10, as well as CelebA and MNIST.
Pros:
- The proposed method is simple and makes intuitive sense. Having editor generators acting like highly-conditioned GANs should make their job easier to produce better samples.
- Empirical results show that when removing editors for evaluation, some well-known architecture (DCGAN - WGAN+GP) can be outperformed with less parameters when comparing IS scores.
- Interesting leads and negative results are discussed in Section 5.
Cons : 
- Comparisons with end-to-end training seems inadequate. The authors invalidate the end-to-end approach with two arguments; (1) that it doesn’t allow for the removal of superfluous editors once the training is done (Section 3.3), and (2) that IS scores are significantly lower (Section 4.2, Table1). It seems that both these statements are true simply because end-to-end learning is performed with a single score outputted by a single critic at the end of the chain, while it would be entirely possible and simple to keep all discriminators and associated losses and train end-to-end. This would still push all editors to produce good samples, thus allowing removal of editors at test-time, and would probably yield better results than those reported in Table 1. It could also invalidate the results shown in Figure 4 (left). For me this is an important missing comparison, as it might even yield better results than the proposed approach and invalidates one of the proposed advantage of the method.
- I think this idea of sequential generation has been explored before, (e.g. StackGAN [1, 2] or LAPGAN [3] and others?), in which unconditional image generation is performed on relatively complicated datasets with a somewhat more principled way of actually simplifying the task of the base generator. Therefore, I think important citations and valid comparisons are missing.
- The only reported metric is the Inception Score (IS), while most of the recent literature agrees that the Fréchet Inception Distance is a better metric. I think FID should be presented as well to be better aligned with recent literature, even in cases where comparison with previously reported performance is impossible (if previous works only presented IS). If you want to be compared to in future work, I think this is necessary.
- It would be a good addition to have FID/IS scores for each editor output, as we could see the quantitative increase in performance at each editing step.
- In Section 4.2, you specify that all experiments are done using the WGAN-GP training formulation. Looking at Table 1 this is unclear, as you specify this training scheme only for model 6 and model 9. If all models use the same training scheme, this information should be absent from the Table.
- CelebA results. Experiments are reported in the main text, without any results, which are only in the Appendix. These results don’t show any quantitative metrics and are visually disappointing. It is hard to see if the editors actually improve the generation.
- Some of the main results or discussions are based on Section 7, which is not the main article even though it is used as another Section instead of an Appendix. I think Section 7 should be separated into Appendices A, B, etc. Maybe some important aspects of the research presented could fit into the main text, given some removal of repetitions, and some compression of the intro to GANs, which should be vastly known by the ICLR community by now.
- Wrong citation format at the end of Section 2.1.
- Section 3.3 : “train the critic more than the generator in each iteration”. This could be clarified by stating exactly what you do (training the critic for k steps for each generator steps).
- It’s not always clear what the boldface represents throughout tables.
- The discussion in Section 5 about promising techniques explored is somewhat disappointing as efforts to investigate why training failed are not apparent.
- Every result shown from the proposed method is performed with ‘small’ or ‘tiny’ versions of existing architectures. This method could have additional value if it could boost performance on the same architecture, even if there are added editors and trainable parameters. The fact that such results are absent makes me suspicious of such a behavior.
Overall I think this paper presents a relevant and interesting idea. However I think this idea has been explored before with more convincing results and with a more principled approach. There are some important flaws in the comparisons made to assess the advantages of the method, and the overall results fail to convince of any important benefit. Based on the pros/cons stated above, I think this paper does not reach the required quality for acceptance in ICLR.
[1] StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks (Zhang et al. 2017)
[2] StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks (Zhang et al. 2017)
[3] Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks (Denton et al. 2015)