Task: Language Identification
Input (Spanish, English): This paper proposes an alternative to Conditional Variational Auto-Encoders and Conditional MultiModal Auto-Encoders to perform inference of missing modalities in dataset with multiple modalities. The proposed approach is a Variational Auto-Encoder jointly on all the modalities  with additional KL divergence penalties between the approximate posterior given all the modalities and the approximate posterior given a subset of the modalities. The approach is named Joint Multimodal Variational Auto-Encoder. The authors make a connection between this approach and the Variation of Information. It is unclear why the authors chose the JMVAE approach instead of a more elegant Variation of Information approach. Another unaddressed issue is the scalability of the method. As far as I can tell (given that no code is provided and the specification of the encoder is missing), this approach requires a new encoder per subset of missing modalities. Right now this approach seems to scale since there are only two modalities. The fact that the estimating the log-likelihood log(p(x)) using multiple modalities provide a lower value than with just one in Table 1 is a bit odd. Could you explain that ? The comparison with between the representation learned by JMVAE and CVAE might be unfair given that the representation of CVAE is learned conditionally, on the label in the case of MNIST, and should therefore not consider the label in this representation. Intuitively, this representation could represent "style" as shown in (Kingma et al., 2014) in their conditional generation figure. For CelebA, comparing log-likelihood on models that use GANs is probably not significant since GAN does not optimizes log-likelihood.  Overall this is an interesting problem and there are also interesting ideas worth exploring further, but the execution of the paper requires more work.
Output (English): en