Venue: ICLR 2018 conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: I really enjoyed reading this paper. Very well-grounded on theory of two-sample tests and MMD and how these ideas can be beneficially incorporated into the GAN framework. It's very useful purely because of its theoretical support of minibatch discrimination, which always seemed like a bit of a hack. So, excellent work RE quality and clarity. One confusion I had was regarding the details of the how the neural embedding feature embedding/kernel \phi(x) is trained in practice and how it affects performance -- must be extremely significant but certainly under-explored. 

I think it's fair to say that the whole paper is approximately minibatch-discrimination + MMD. While this is a very useful and a  much more principled combination, supported by good experimental results, I'm not 100% sure if it is original enough. 

I agree with the authors that discriminator "overpowering" of generators is a significant issue and perhaps a little more attention ought to have been given for the generators being more effective w.r.t. 2S tests and distributional discrimination, as opposed to the regularization-based "hack". 

I would've also liked to have seen more results, e.g. CIFAR10 / SVHN. One of the best ways to evaluate GAN algorithms is not domain adaptation but semi-supervised learning, and this is completely lacking in this paper. 

Overall I would like to see this paper accepted, especially if some of the above issues are improved. 
review_title: Good and clear paper, well-written, but potentially too incremental

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper looks at the problem of mode collapsing when training GANs, and proposes a solution that uses discriminators to compare distributions of samples as opposed to single observations. Using ideas based on the Maximum Mean Discrepancy (MMD) to compare distributions of mini-batches, the authors generalize the idea of mini batch discrimination. Their tweaked architecture achieves nice results on extensive experiments, both simulated and on real data sets. 

The key insight is that the training procedure usually used to train GANs does not allow the discriminator to share information across the samples of the mini batch. That is, the authors write out the gradient of the objective and show that the gradient with respect to each observation in the batch is multiplied by a scaling factor based on the discriminator, which is then summed across the batch. As a result, this scaling factor can vanish in certain regions and cause mode collapsing. Instead, the authors look at two different discrepancy metrics, both based on what they call neural mean embeddings, which is based on MMD. After describing them, they show that these discriminators allow the gradient weights to be shared across observations when computing gradients, thus solving the collapsing mode problem. The experiments verify this. 

As the authors mentioned, the main idea is a modification of mini batch discrimination, which was also proposed to combat mode collapsing. Thus, the only novel contributions come in Section 3.1, where the authors introduce the mini-batch discriminators. Nevertheless, based on the empirical results and the coherence of the paper (along with the intuitive gradient information sharing explanation), I think it should be accepted. 

Some minor points: 
-How sensitive is the method to various neural network architectures, initializations, learning rates, etc? I think it's important to discuss this since it's one of the main challenges of training GANs in general.
-Have you tried experiments with respect to the size of the mini batch? E.g. at what mini batch size do we see noticeable improvements over other training procedures?
-Have you tried decreasing lambda as the iteration increases? This might be interesting to try since it was suggested that distributional adversaries can overpower G particularly early in training.
-Figure 1 is interesting but it could use better labelling (words instead of letters)

Overall:
Pros: Well-written, good empirical results, well-motivated and intuitively explained
Cons: Not particularly novel, a modification of an existing idea, more sensitivity results would be nice
review_title: Improving GAN training comparing samples instead of observations

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper proposes to replace single-sample discriminators in adversarial training with discriminators that explicitly operate on distributions of examples, so as to incentivize the generator to cover the full distribution of the training data and not collapse to isolated modes. 

The idea of avoiding mode collapse by providing multiple samples to the discriminator is not new; the paper acknowledges prior work on minibatch discrimination but does not really describe the differences with previous work in any technical detail. Not being highly familiar with this literature, my reading is that the scheme in this paper grounds out into a somewhat different architecture than previous minibatch discriminators, with a nice interpretation in terms of a sample-based approximation to a neural mean embedding. However the paper does not provide any empirical evidence that their approach actually works better than previous approaches to minibatch discrimination. By comparing only to one-sample discriminators it leaves open the (a priori quite plausible) possibility that minibatch discrimination is generally a good idea but that other architectures might work equally well or better, i.e., the experiments do not demonstrate that the MMD machinery that forms the core of the paper has any real purchase.

The paper also proposes a two-sample objective DAN-2S, in which the discriminator is asked to classify two sets of samples as coming from the same or different distributions. This is an interesting approach, although empirically it does not appear to have any advantage over the simpler DAN-S -- do the authors agree with this interpretation? If so it is still a worthwhile negative result, but the paper should make this conclusion explicit. Alternately if there are cases when the two-sample test is actually recommended, that should be made explicit as well. 

Overall this paper seems borderline -- a nice theoretical story, grounding out into a simple architecture that does seem to work in practice (the domain adaptation results are promising), but with somewhat sloppy writing and experimentation that doesn't clearly demonstrate the value of the proposed approach. I hope the authors continue to improve the paper by comparing to other minibatch discrimination techniques. It would also be helpful to see value on a real-world task where mode collapse is explicitly seen as a problem (and/or to provide some intuition for why this would be the case in the Amazon reviews dataset). 

Specific comments:
- Eqn (2.2) is described as representing the limit of a converged discriminator, but it looks like this is just the general gradient of the objective --- where does D* enter into the picture?
- Fig 1: the label R is never explained; why not just use P_x?
- Section 5.1 "we use the pure distributional objective for DAN (i.e., setting λ != 0 in (3.5))" should this be λ = 0? 
- "Results" in the domain adaptation experiments are not clearly explained -- what do the reported numbers represent? (presumably accuracy(stddev) but the figure caption should say this). It is also silly to report accuracy to 2 decimal places when they are clearly not significant at that level.
review_title: interesting ideas, would like more empirical support

Meta-review: All the reviewers and myself have concerns about the potentially incremental nature of this work. While I do understand that the proposed method goes beyond crafting minibatch losses, and instead parametrizes things via a neural network, ultimately it's roughly very similar to simply combining MMD and minibatch discrimination and "learning  the kernel". The theoretical justifications are interesting, but the results are somewhat underwhelming (as an example, DANN's are by no means the state of the art on MNIST->MNIST_M, and this task is rather contrived; the books dataset is not even clearly used by anyone else).

The interesting analysis may make it a good candidate for the workshop track, so I am recommending that.
Decision: Invite to Workshop Track