Venue: ICLR 2018 conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: 
I thank the authors for the thoughtful responses and updated manuscript. Although the manuscript is improved, I still feel it is unfocused and may be substantially improved, thus my review score remains unchanged.

===============

The authors describe a new version of a generative adversarial network (GAN) for generating images that is heavily related to class-conditional GAN's. The authors highlight several additional results on evaluation metrics and demonstrate some favorable analyses using their new proposed GAN.

Major comments:
1) Unfocused presentation. The paper presents a superfluous and extended background section that needs to be cut down substantially. The authors should aim for a concise presentation of their work in 8 pages. Additionally, the authors present several results (e.g. Section 5.1 on dynamic labeling, Section 6.1 on Inception score) that do not appear to improve the results of the paper, but merely provide commentary. The authors should either defend why these sections are useful or central to the arguments in the paper; otherwise, remove them.

2) Quantitative evaluation highlight small gains. The gains in Table 1 seem to be quite small and additionally there are no error bars so it is hard to assess what is statistically meaningful. Table 2 highlights some error bars but again the gains some quite small. Given that the AM-GAN seems like a small change from an AC-GAN model, I am not convinced there is much gained using this model.

3) MS-SSIM. The authors' discussion of MS-SSIM is fairly confusing. MS-SSIM is a measure of image similarity between a pair of images. However, the authors quote an MS-SSIM for various GAN models in Table 3. What does this number mean?  I suspect the authors are calculating some cumululative statistics across many images, but I was not able to find a description, nor understand what these statistics mean.

4) 'Inception score as a diversity measurement.' This argument is not clear to me. Inception scores can be quite high for an individual image indicating that the image 'looks' like a given class in a discriminative model.  If a generative model always generates a single, good image of a 'dog', then the classification score would be quite high but the generative model would be very poor because the images are not diverse. Hence, I do not see how the inception score captures this property.

If the authors can address all of these points in a substantive manner, I would consider raising my rating.


review_title: Review of Activation Maximization

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: This paper is a thorough investigation of various “class aware” GAN architectures. It purposes a variety of modifications on existing approaches and additionally provides extensive analysis of the commonly used Inception Score evaluation metric.

The paper starts by introducing and analyzing two previous class aware GANs - a variant of the Improved GAN architecture used for semi-supervised results (named Label GAN in this work) and AC-GAN, which augments the standard discriminator with an auxiliary classifier to classify both real and generated samples as specific classes. 

The paper then discusses the differences between these two approaches and analyzes the loss functions and their corresponding gradients. Label GAN’s loss encourages the generator to assign all probability mass cumulatively across the k-different label classes while the discriminator tries to assign all probability mass to the k+1th output corresponding to a “generated” class. The paper views the generators loss as a form of implicit class target loss.

This analysis motivates the paper’s proposed extension, called Activation Maximization. It corresponds to a variant of Label GAN where the generator is encouraged to maximize the probability of a specific class for every sample instead of just the cumulative probability assigned to label classes. The proposed approach performs strongly according to inception score on CIFAR-10 and includes additional experiments on Tiny Imagenet to further increase confidence in the results.

A discussion throughout the paper involves dealing with the issue of mode collapse - a problem plaguing standard GAN variants. In particular the paper discusses how variants of class conditioning effect this problem. The paper presents a useful experimental finding - dynamic labeling, where targets are assigned based on whatever the discriminator thinks is the most likely label, helps prevent mode collapse compared to the predefined assignment approach used in AC-GAN / standard class conditioning.

I am unclear how exactly predefined vs dynamic labeling is applied in the case of the Label GAN results in Table 1. The definition of dynamic labeling is specific to the generator as I interpreted it. But Label GAN includes no class specific loss for the generator. I assume it refers to the form of generator - whether it is class conditional or not - even though it would have no explicit loss for the class conditional version. It would be nice if the authors could clarify the details of this setup.

The paper additionally performs a thorough investigation of the inception score and proposes a new metric the AM score. Through analysis of the behavior of the inception score has been lacking so this is an important contribution as well.

As a reader, I found this paper to be thorough, honest, and thoughtful. It is a strong contribution to the “class aware” GAN literature.

review_title: Thorough investigation and extension of class-aware GAN approaches

final_score: rating: 7: Good paper, accept
review_content: + Pros:
- The paper properly compares and discusses the connection between AM-GAN and class conditional GANs in the literature (AC-GAN, LabelGAN)
- The experiments are thorough
- Relation to activation maximization in neural visualization is also properly mentioned
- The authors publish code and honestly share that they could not reproduce AC-GAN's results and thus using to its best variant AC-GAN* that they come up with. I find this an important practice worth encouraging!
- The analysis of Inception score is sound.
+ Cons:
- A few presentation/clarity issues as below
- This paper leaves me wonder why AM-GAN rather than simply characterizing D as a 2K-way classifier (1K real vs 1K fake).

+ Clarity: 
The paper is generally well-written. However, there are a few places that can be improved:
- In 2.2, the authors mentioned "In fact, the above formulation is a modified version of the original AC-GAN..", which puts readers confusion whether they were previously just discussed AC-GAN or AC-GAN* (because the previous paragraph says "AC-GAN are defined as..".
- Fig. 2: it's not clear what the authors trying to say if looking at only figures and caption. I'd suggest describe more in the caption and follow the concept figure in Odena et al. 2016.
- A few typos here and there e.g. "[a]n diversity measurement"

+ Originality: AM-GAN is an incremental work by applying AM to GAN. However, I have no problems with this.
+ Significance: 
- Authors show that in quantitative measures, AM-GAN is better than existing GANs on CIFAR-10 / TinyImageNet. Although I don't find much a real difference by visually comparing of samples of AM-GAN to AC-GAN*.

Overall, this is a good paper with thorough experiments supporting their findings regarding AM-GAN and Inception score!
review_title: good paper with thorough experiments!

Meta-review: The authors investigate various class aware GANs and provide extensive analysis of their ability to address mode collapse and sample quality issues. Based on this analysis they propose an extension called Activation Maximization-GAN which tries to push each generated sample to a specific class indicated by the Discriminator. As experiments show, this leads to better sample quality & helps with mode collapse issue. The authors also analyze inception score to measure sample quality and propose a new metric better suited for this task.
Decision: Accept (Poster)