Venue: ICLR 2017 conference

Reviews:

final_score: rating: 3: Clear rejection
review_content: The paper proposed conditional biGAN and its extension to multi-view biGAN. The main idea of conditional biGAN is to matching the latent variable distributions of two encoders, each of which are conditioned on the observation (\tilde{x}) and the output (y), respectively, in addition to standard biGAN formulation.

The description on MV-GAN require more revision. Specifically, the definition on aggregating model, \Phi, mapping v and a variable s should be clarified. Looking at Equation (8), I can't find a term that constrains the output domain of function v to be the same as a data domain. 

Experimental results are not convincing. In most generation results, the observation is not very well preserved. For example, in Figure 6 the second row, background changes significantly from the observation. We also observe such behavior in digit generation example. Preserving attributes like gender is interesting but doesn't seem to be a strong indication that the model learn to correlate observation and the output through latent variable. 
review_title: null

final_score: rating: null
review_content: I have a few questions regarding this paper:

1) Does n_k refer to the number of points observed for view k, or to the number of features with which each point is represented? 

2) Why is \phi used? What is the benefit from the basic approach where we let the latent space z deal with the representation of \tilde{x}? Also, how is \phi parameterized? Doesn't the consideration of \phi (instead of just having \tilde{x}) make training more tricky (i.e. it'd be easier to learn p(z|\tilde{x}) rather than p(z | \Psi))?

3) I'd be interested to see more discussion on CGAN vs CV-BiGAN; from the current discussion it seems that CV-BiGAN is like CGAN but with an extra mapping (following BiGAN). Is there a further distinction?

Thank you
review_title: A few questions

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper builds in the bidirectional GAN (BiGAN) to obtain an extension which can handle multiple views of data. To this end, the authors extend the BiGAN for multiple view aggregation. This is an easy task and just requires the introduction of the additional distribution and accompanying discriminator. The main challenging and novel part is in regularizing the model to avoid instabilities. The authors propose a novel KL divergence based constraint.

As mentioned above, the approach builds quite heavily on previous ones but it has enough novel elements, in particular the constraint for regularization. This constraint is a reasonable assumption an in practice seems to work well. One downside is that it comes with a parameter \lambda which controls its strength and which is not obvious how to find efficiently. For example, for the MNIST data it takes a very small value, 10^-5 and for the CELEBA it's 10^-3. It could be that the results are not very sensitive to this value, but there's no discussion concerning this aspect. 

Apart from the regularization term, the rest of the model construction is well motivated, I agree with the authors that a multi-view approach employing GANs is an interesting topic to consider. 

Presentation is in general good although at parts readability is hindered. I feel that the notation is unnecessarily complicated, and some parts of the text too. Furthermore, it wasn't immediately obvious to me what is considered as \tilde{x} and what is y in the experiments. 

Additionally, most of the discussion on the well-studied area of multi-view learning (intro and sec. 6) omits reference to important prior work which is not based on neural networks. Indeed, some of the issues mentioned as common in today's methods (discrete outputs only, no density estimation...) do not actually exist in many non-neural network approaches. There are too many works to suggest including in the discussion, but I guess the most relevant ones are in the field of probabilistic and non-linear multi-view learning, which is also what MV-BiGAN is doing.

The experiments presented in the paper are nice illustrations but unfortunately insufficient. Firstly, they only cover the task of generating (small) images and correspond to non-real world settings (I'd actually consider all of the experiments as toy experiments). Furthermore, the most difficult of these experiments (sec. 5.3) is quite unconvincing. If Fig. 6 shows some of the best results that can be achieved, then this is rather disappointing. Important details are also missing: what is exactly the attribute vector used? How many instances exist?

The two other experiments on 5.1 and 5.2 are well executed. It's important that the authors show a comparison with \lambda=0. 

However, beyond showing the validity of the KL term, one can't conclude much about the overall merit of the method as a multi-view learning approach, given the above experiments. 

Overall this was a nice paper to read, but seems somehow incomplete.

review_title: nice paper but seems somehow incomplete

final_score: rating: null
review_content: I have not found major problems with the formulation, derivation etc. 

At this point, I am curious, have you used the latent low-dimensional features for some downstream tasks? A quick scan indicates the authors are mostly focusing on generation tasks.

It would be good to consider using those features for the multi-view representation learning tasks, such as those in the deep variational CCA paper. 
review_title: have you used the latent features for some tasks

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper presents extensions of bidirectional generative adversarial networks to the conditional setting and multi-view setting. The methods are well-motivated, the mathematical derivations appear to be correct, and the presentation is clear enough to me. 

I would suggest this paper to be accepted. However, I find it somewhat limited to only present results for generation tasks. I think a main advantage of using bi-GAN (rather than the standard GAN) is the additional inference model that can learn useful features. I am curious about how good the features are for some other supervised (or semi-supervised) learning tasks and what have they really learned.

I also find it interesting that the counterpart of these models under the VAE framework have also been proposed
- Kihyuk Sohn and Honglak Lee and Xinchen Yan. Learning Structured Output Representation using Deep Conditional Generative Models. NIPS 2015.  (*** contitional VAE ***)
- Weiran Wang, Xinchen Yan, Honglak Lee, and Karen Livescu. Deep Variational Canonical Correlation Analysis. In submission to ICLR 2017. (*** sort of multi-view VAE ***)

It would be nice to have discussions and comparisons in future work. 
review_title: nice extensions to Bi-GAN

Meta-review: The presented approach builds heavily on recent work, but does provide some novelty. The presentation is generally all right, but there are parts of the manuscript that the reviewers feel needed/needs work. All reviewers note that the evaluation and experimental work could be improved.
Decision: Reject