Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: The authors extend the VaDE model (Jiang et al.) to allow for multiple, independent facets i.e. Mixtures of Gaussians. The authors first point out and resolve a technical oversight in derivations related to estimation of $q(c \mid x)$ of Jiang et al. (VaDE). The authors implement their formulation in an architecture based on ladder VAE to extract multiple clusterings of the data simultaneously. The authors demonstrate their method on 3 different benchmark datasets.
main_review: Note: I'm not very familiar with other comparable models learning hierarchical structure in latent variables, and cannot judge the novelty of the contribution too well. I'm willing to revise my score in case I have missed something. 

Pros
 - The paper is clearly written, and the theoretical portion is well laid out and pleasant to read. The ideas are uncluttered, and the appendices provide enough context for an outsider to the field to follow. 

Cons:
1. The authors do not learn more than a handful of clusters in each facet, and do not practically demonstrate the feasibility of the approach in a setting where the number of independent clusters blows up. It might be helpful to elaborate more on the shortcomings of Gumbel-softmax based approaches or even compare this approach with one where the discrete variable is learned through the re-parametrization trick.

2. Many details have been offloaded to the appendices. This is not a major issue since the appendices are well written. For example, it might might help to move schematics and related description for the ladder MFCVAE to the main text since this is the version used for all 3 datasets/ to demonstrate the key points in Sec 4.1-4.4.

Other comments:
 - It's not clear how much do results depend on various hyperparameter choices. In particular, number of components $K$ for each facet, dimensionality of $z$, priors on the distribution across components, are likely central to the interpretability/ apparent disentaglement of the models. It would be great if the authors can address this point.

 - Majority of the demonstrations rely on qualitative assessment of the reconstructions, for visual datasets where the factors have some semantic meaning (e.g. color, stroke styles, digit identity etc.), and convolutional neural network layers are known to compose features into images. It would be great if the authors could demonstrate these methods for exploratory settings, for example to make predictions on publicly available tabular, biological datasets that come with rich metadata and often a hierarchical structure. 
limitations_and_societal_impact: The paper and appendices include thoughtful notes for each dataset. I also appreciate the transparency in discussing limitations and experimental details!
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper proposed Multi-Facet Clustering Variational Autoencoders, which is an unsupervised model and can learn multiple latent characteristics of the data. They showed how to optimize the ELBO in this setting and implemented the model via Variational Ladder Autoencoder architecture with progressive training. Experimental results demonstrate good empirical performance.

main_review: Real-world data usually can be clustered in multiple ways. However, most existing deep learning based clustering algorithms are designed to learn only one such partition. This paper aims to learn multiple levels of abstract characteristics that in combination can form different partitions of data. They assume Mixtures of Gaussians prior on the latent space. The paper extends variational deep embedding to this multi-facet setting and shows how to optimize the ELBO. The experiments are done on datasets of MNIST, 3DShapes, and SVHN. While the experiments seem thorough, the performance comparison to other methods was not clearly presented.

While the concept seems to be meaningful, it is unclear to me how practical these (independent) facets are. Not every data set clearly has distinct characteristics like handwritten numbers. During implementation, it’s hypothesized that each layer of the VLAE captures distinct aspects of the data. If so, how many layers are needed? Are more layers better? Is it possible that some layers learn combinations of characteristics? More discussions on these aspects will be appreciated.

The paper is in general well written and the flow is easy to follow. Much of the experiments are postponed to the appendix, making that section difficult to read.

limitations_and_societal_impact: Yes.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper presents Multi-Facet Clustering Variational Autoencoder which is a probabilistic model base on multiple mixtures of Gaussians that find multiple characteristics of the data simultaneously. Moreover, the paper shows theoretical results for the optimisation of ELBO analytically with respect to the categorical variational posterior distribution.
main_review: Originality:
The authors of this paper deal with a known problem of clustering a multi-facet that is less common than single-facet models. In the proposed method, they use well-known techniques such as Monte Carlo, progressive training, a mixture of Gaussians, or estimating the ELBO but it is a novel combination of them. The novel theoretical results for optimising its ELBO is extending an influential related paper for the single-facet case. 

Quality and clarity:
In general, the paper is a complete piece of work but section Experiments needs some refinement. The authors show the results of the proposed method MFCVAE and severals samples generated from MFCVAE trained on MNIST, 3DShapes, and SVHN. In my opinion, quantitative results are lacking compared to other methods. 
The paper was presented classification performance in terms of unsupervised clustering accuracy on the test set, comparing commonly used single-facet and multi-facet deep clustering approaches. These results show the average effectiveness of the proposed method, the authors emphasize that high classification accuracy was not their goal. A table would be a better form of presenting these results. It is not clear why actors do not use the same comparative methods for different datasets.

Significance:
The paper presents a new approach to a less-known problem of clustering a multi-facet. An important result of this work is novel theoretical results for optimising its ELBO analytically concerning the categorical variational posterior distribution. This result may be used by other researchers. Unfortunately, the evaluation of the proposed method requires additional work (I have given some of the comments above). The paper presents only the results for J = 2, while for most of the data sets J is much greater than 2. The positive fact is that the authors know about it, they indicate that this problem will be solved in the future.

limitations_and_societal_impact: yes
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: There are multiple interesting aspects of high-dimensional data.
This paper proposes a Multi-Facet Clustering Variational Autoencoders (MFCVAE) to cluster the data in multi-aspects.
MFCVAE utilizes and extends the trick of VaDE, and provides the methods to learn multi-aspect clustering in an end-to-end fashion.
This paper provides various qualitative results to validate the multi-aspect clustering performance.
main_review: < originality >
1. MFCVAE utilizes a mixture of Gaussian prior for multiple clustering simultaneously, and the task is important and novel.
2. For simultaneous clustering, MFCVAE utilizes and extends the VaDE tricks. The extension is somewhat novel.
3. MFCVAE is related to disentangled representation, but this paper does not discuss the relationship between MFCVAE and various disentanglement models, [1,2,3]. Discussion on the relationship between MFCVAE and various disentanglement methods will be interesting

< quality >
1. The technical part is sound
2. The paper does not provide the quantitative methods to measure the disentanglement and sample diversity. The lack of quantitative measure makes the reader not ensure the effectiveness of MFCVAE

< clarity >
1. The paper is well-written.
2. It is hard to clarify the quantitative experimental results. There is no quantitative results table.
3. There exist only J=2 experiments, and additional experimental results for J>2 will be more interesting.

< significance >
1. The experimental results of MFCVAE are not significant. On MNIST, MFCVAE shows 92.02%, but IMSAT achieves 98.4%.
2. There is no quantitative diversity metric, and it is hard to ensure that the MFCVAE generates better diverse samples.

[1] Kim, Hyunjik, and Andriy Mnih. "Disentangling by factorising." International Conference on Machine Learning. PMLR, 2018.
[2] Mathieu, Emile, et al. "Disentangling disentanglement in variational autoencoders." International Conference on Machine Learning. PMLR, 2019.
[3] Sha, Lei, and Thomas Lukasiewicz. "Multi-type Disentanglement without Adversarial Training." Proceedings of the 35th AAAI Conference on Artificial Intelligence. 2021.
limitations_and_societal_impact: The authors adequately addressed the limitations of their work.
review_title: null

Meta-review: The paper is well-written and the proposed method shows good performance (quantitative results were shown in the rebuttal).  Correcting a flaw in previous work is a plus.  Reviewer's major concerns, including lack of quantitative results, dependence on hyperparameter choice, etc., have been addressed in the rebuttal. 
Decision: Accept (Poster)