Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors propose to use contrastive learning for matching in latent space in the Wasserstein autoencoder (WAE). In addition, they employ techniques such as momentum contrast in contrastive learning. Experimental results show that the proposed method, MoCA, is more stable and converges faster than existing methods. It is also capable of generating high-resolution images such as CelebA-HQ.

main_review: Strengths:
- Overall, this paper is well organized.
- The method proposed in this paper is a reasonable combination of existing state-of-the-art methods.
- Experimental results show that the proposed method has stability and faster convergence, which is promising.

Weaknesses:
- The authors claim that MoCA can generate images with high quality. However, the experimental results in this paper do not show this very well. First of all, although the authors claim that the results in Figure 5 and Figure 6 are "look realistic", some of the face images seem to be collapsed, and the interpolation between the two images seems to be discontinuous. Since there is no qualitative comparison with existing methods, we cannot judge these methods as "realistic". In Table 1, the authors show the quantitative comparison results with the existing methods, but there are some puzzling points. First, why does MoCA-A2, which has fewer parameters, have higher performance? Also, why does WAE-GAN perform better than MoCA? Since Table 1 shows that WAE-GAN is better than MoCA, shouldn't it be compared with WAE-MMD? Furthermore, why are the quantitative results of CelebA-HQ not shown?
- The authors show in their experiments that MoCA achieves faster convergence than existing methods, but they do not fully explain why MoCA shows such convergence. Why does the convergence of the proposed method become better when contrastive learning is included? In addition, Figure 1 shows the line graphs of only one training trial for each method, and the variance of each method is not shown. Therefore, I cannot judge whether the difference in results between methods is large or small.
- In Section 4.1, the authors compare WAE-MMD with MoCA as the original WAE algorithm, but I do not understand why they do not compare it with WAE-GAN, which is also original. I think this should be done because Table 1 shows that WAE-GAN has better image generation performance than the proposed method.
 For example, the authors employ MoCo, but how effective is this in improving the performance of the proposed method? The authors do not seem to have verified such a thing.

Minor comments:
 - The significant figures of the results of each method in Table 1 should be the same.
 - Section3: any fixed t -> any fixed \tau
summary_of_the_review: In terms of convergence and stability, the proposed method is considered to be effective to a certain extent. Also, the idea of using contrastive learning for WAE is interesting. However, the explanation of the claim and the presentation of the results are insufficient.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper presents a regularization technique for Wasserstein Auto-Encoders, based on contrastive learning. 
main_review: Strengths:

1. The paper is well-written and easy to follow. I do think that some of the notation such as "push forward" from measure theory, is really not needed or particularly useful here. Simpler terminology such as just using encoding and decoding functions would be more than sufficient.

2. Some of the experiments are interesting and show the effects of the proposed regularization e.g. on the singular value distribution of the latent representation.

3. Using a contrastive approach is a potentially effective way to match the prior and posterior distributions.

Weaknesses: It is unclear that the proposed regularizer results in qualitatively better reconstructions than the baselines. FID is not a perfect measure and the samples from baselines should be shown side-by-side with the proposed approach to know whether there is indeed an improvement. I found the CIFAR-10 reconstruction results are somewhat poor. 

Question: the value of the lambda parameter is very large - what are the relative loss values during training/convergence (the reconstruction loss vs. regularizer loss)?
summary_of_the_review: The paper is interesting but has some shortcomings. I would like to see some results of the baselines to decide if the proposed regularizer does indeed improve results qualitatively. I do not believe that FID is a proper measure of quality (not just for this paper but for measurement of GAN sample quality, in general). I give the paper a slightly positive score based on the idea, but I am looking forward to some samples in the rebuttal to decide my final score.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a new approach to train Wasserstein auto-encoders (WAE) with contrastive learning techniques. Specifically, the paper proposes to enforce the marginal matching constraint of WAE by exploiting the fact that contrastive learning objectives optimize the latent space distribution to be uniform over the unit hypersphere.
main_review: I notice this is a re-submission from ICLR-2021. Thus some of my comments are based on the differences between two versions.

## Strengths
1. The paper is well written and well motivated.
2. I think the idea of using contrastive learning to enforce a hypersphere prior for WAE is clever and neat.
3. The authors provide extensive ablations on hyperparameters.


## Weaknesses
1. My main concern is the performance of the proposed method on CIFAR10 and CelebA. The interpolation, reconstruction, and samples in Figure 6 are very blurry, and hard to justify the benefit of using the proposed approach. The reported FID in Table 1 and 2 are very high. It would be nice to include a comparison of [1] (which has FID of 5.25 and 24.08 on CelebA and CIFAR10 respectively). Also, why is the two-stage VAE baseline in the previous version removed?
2. It would be nice to include WAE-GAN in Figure 1 and 2, since it outperforms the proposed MoCA in Table 1.
3. I think it would be interesting to see how to integrate the instance contrastive loss as in DC-VAE [2] into the proposed MoCA.


[1] Aneja, Jyoti, et al. "Ncp-vae: Variational autoencoders with noise contrastive priors." arXiv preprint arXiv:2010.02917 (2020).
[2] Parmar, Gaurav, et al. "Dual contradistinctive generative autoencoder." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.
summary_of_the_review: The main idea of the paper is well motivated. However, I still find the results on image datasets such as CIFAR10 and CelebA hard to justify the superiority of the proposed method over baselines. I lean towards weak rejection but am willing to amend my score if my concerns are addressed.
review_title: null

Meta-review: This paper presents a variant of the WAE which uses a contrastive criterion to enforce the marginal distribution matching constraint. Experiments show faster convergence in terms of Wasserstein distance, more visually appealing samples, and better FID scores compared with other WAE models.

The original WAE framework leaves open the choice of approximation for enforcing marginal distribution matching, and the original paper gives two such algorithms. Therefore, it's pretty natural to replace this approximation with something else (such as the contrastive criterion used here), so a submission would need to show evidence that it's significantly better than other approaches. Reviewers have expressed various concerns about the experiments. None of them are major problems, but overall the method doesn't seem consistently better than other WAE methods; e.g., the FID score is worse than that of WAE-GAN.

I encourage the authors to take the reviewers' comments into account in preparing the submission for future cycles.
Decision: Reject