Venue: UAI 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate to high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations. 
review_content: Q1 Summary and contributions: This paper proposes a deep Dirichlet process mixture (DDPM) model to jointly achieve clustering and feature learning in an unsupervised fashion. This method combines the strengths of the traditional DPM models and deep neural networks, and thus it inherits the ability of DPM to adapt the number of mixture components, and also has the ability to learn complex and nonlinear features.
Q3 Main strengths: [1] The whole structure of this manuscript is easy to follow and understand with good organization. The given method with the source-adaptive idea is novel in the field of the Dirichlet process. 

[2] Regarding the reproducibility of this given method, it is easy to implement since the code is available and there are sufficient experimental details; thus, I think it has good credibility and reproducibility. 
Q4 Main weakness: Some motivations are not well explained. 
Q5 Detailed comments to the authors: [1] Some motivations are not well explained. For example, as mentioned in the introduction that selecting the number of clusters is important,  and as far as I know that there are many methods to select the number of clusters, such as BIC, AIC, or some other criteria, why does this work use the Dirichlet process mixture model to solve this problem?

[2] Will the invertibility of the nonlinear neural network f reduce the representability of the network? Could you provide some discussion on this point?

[3] Since the proposed algorithm 1 adopts the Gibbs Sampling method which has been recognized to be slow,  how about the computational efficiency of this algorithm?
Q2(4) Quality of experiments (Optional): 3: Good: The experimental evaluation is adequate, and the results convincingly support the main claims.
Q7 Justification for your score: Novelty is the most important factor in my rating.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper with, with novel ideas, excellent impact on at least one area of AI or high to excellent impact on multiple areas of AI, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations. 
review_content: Q1 Summary and contributions: The paper proposes a clustering method combining Dirichlet processes on Gaussian mixtures and NICE flow-based feature learning. Learning is via MCEM. Experimental results on medium-to-large size data sets demonstrates the usefulness of this method.
Q3 Main strengths: This is a novel combination of DP and NICE-based feature learning, initialised with VAE. In addition, the learning is via MCEM, which is effective.

The generated examples in Figure 4 also shows clearly that this method works very well.
Q4 Main weakness: - The F score is not really a useful metric for a clustering problems.

- The last statement in section 4.2.3 and Figure 3 are not supported. For example, clusters for 7 and 9 does not really get "denser and more concentrated".

Q5 Detailed comments to the authors: - Abstract: The term "capacity" may be better replaced by "flexibility"

- Section 1, 2nd para: there should be a better word than "reversely" .

- Section 1, 3rd para: ditto for "insufficiency"

- Section 3.2, 2nd para: $x$ and $y$ could be replaced by $y$ and $z$ to put it in the same context of eq (3) and preceding derivations.

- Footnote 1: scalar -> vector

- NG in equation 7 not defined.

- Second column heading for Table 2 is not appropriate.

- Figure 2 subcaption: advance -> advantage

- Figure 2 should have better  elaboration.

- Figure 2: K-means on the AE and flow-learned features is better than DDPM. This is worth pointing out and further explained.

-  [Dempster, 1977] is a better reference for EM.

- "The EM Algorithm and Extensions" is a better reference for MCEM.
Q2(4) Quality of experiments (Optional): 3: Good: The experimental evaluation is adequate, and the results convincingly support the main claims.
Q7 Justification for your score: Overall novel combination of existing advanced methods. The weaknesses can be easily fixed.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., lack of novelty, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: Q1 Summary and contributions: This introduces 'Deep Dirichlet Process Mixture Model', which extends the Dirichlet process model to large datasets. The model is comprised of a fixed pre-trained autoencoder projecting the dataset down to ten dimensions, followed by a normalizing flow. The results are then used as inputs to a Dirichlet Process Model. The model's parameters are learned by expectation maximization (EM). This allows deep clustering with a nonparametric model over the number of clusters, $K$
Q3 Main strengths: The paper is clearly written and the method is described well. The experiments fairly indicate the strengths of the model, as well as potential weaknesses. Figure 1 is a particularly nice diagram of how the model works. 
Q4 Main weakness: Unclear extent to which the DPM portion is having an effect:
The paper claims that the DDPM method is an effective deep clustering method for high dimensions. However, it seems that most of the clustering power in the approach comes from the autoencoder, which projects the data down to ten dimensions in all cases.

Limited Evaluation and Ablations:
The authors compare only against the G-means and DPM approach for the experiments. I believe there are several recent approaches in deep unsupervised clustering, as discussed in the related work. It would be instructive to compare DDPM against these methods when viewing $K$ as a hyperparameter. 

Missing Related Work?
The paper seems to be missing any discussion to [Echraibi, 2020, 'On the Variational Posterior of Dirichlet Process
Deep Latent Gaussian Mixture Models']. This work discusses an approach that seems quite similar, with a Dirichlet Process model followed by a projection to the data space.
Q5 Detailed comments to the authors: Unclear extent to which the DPM portion is having an effect:
I think it's necessary to see an ablation where the autoencoder is made less powerful and the behaviour of the overall model is examined. In particular, figure 3 seems to show that the encoder has achieved almost complete clustering even before any of the DDPM model runs. (Of course a t-SNE can be misleading in that particular case, but I think the authors need to address this general point). The authors also present the method as a combination of feature learning via the normalizing flow and cluster identification with the DPM, but judging by the MNIST example, the vast majority of the feature learning seems to be done in the pre-training, where the DPM is not involved.

Limited Evaluation and Ablations:
What if you choose the ground truth $K$ as the input for these clustering methods? What if $K$ is slightly mispecified, or you try a couple of values of $K$? What if you have the same computational budget available to DDPM and competing deep clustering methods? These are the sort of questions that an effective experimental comparison could answer.

In fact, DDPM also has hyperparameters to choose, such as the $\alpha_0$ value in table 1, which varies across seven orders of magnitude. How is this chosen? 

Q2(4) Quality of experiments (Optional): 3: Good: The experimental evaluation is adequate, and the results convincingly support the main claims.
Q7 Justification for your score: The fact that this method relies so heavily on a pre-trained encoder is the main driving force behind the low score. The paper claims to combine deep feature development with cluster identification and separation with a DP. But it only ever operates on a ten-dimensional space heavily pre-processed by the autoencoder. If the method could be combined with a NF on pixels (there are many recent examples), this could be an exciting method. As described, the current method is of limited significance.
review_title: null

Meta-review: Meta Review: The paper proposes a non-linear clustering algorithm, based on a Dirichlet Process Mixture Model as a latent variable model in the latent space of a normalizing flow. In that way, the model automatically learns a suitable number of clusters, simultaneously with a non-linear feature extractor. The reviews were largely positive and I recommend acceptance as a poster.

However, one reviewer raised the point that for pre-processing an autoencoder was used. The results are thus somewhat inconclusive, since we observe the combined effect of autoencoder and the proposed method. I agree with this criticism. I want to strongly encourage the authors to include an ablation study (i.e. run the experiments without autoencoder) so that the contribution of autoencoder and DPM+flow model becomes delineated.

Pros:
Well written, reproducible, good experimental results

Cons:
Experiments not fully conclusive, limited ablations, potentially missing related work 

Quality: Good
Clarity: Good 
Originality: Fair
Significance: Fair but unclear to some extent

Decision: Accept (Poster)