Paper: Interpretable Deep Clustering (ICLR-2024)

Reviewer comment (Writing):
The motivation is not clear, I don not understand what is interpretable clustering model and why we need interpretable clustering model.

Author response:
P1 Motivation of interpretable clustering-

The second paragraph in the introduction motivates our interpretable clustering method. To elaborate on the problem, we offer the following paragraph:

The main goal of clustering methods is to partition the data into semantically related groups. In some use cases, it is sufficient to assess the separability of the clusters to evaluate the clustering quality. However, clusters are used in many applications for downstream tasks, such as scientific discovery, decision-making, medical analysis, and more. In such cases, cluster assignment interpretations are crucial for annotating the clusters manually. Cluster interpretability is also vital for quality assessment, model validation, and gaining trust in practitioners. 

For example, let’s consider the task of cell clustering (also mentioned in the introduction). Common practice is to use clustering and then manually analyze each cluster to identify which cell types it represents. Cell annotation is vital for downstream tasks, such as drug discovery, personalized treatment, and automated diagnosis and prognosis. In these cases, identifying cluster-level and sample-level informative features is crucial for labeling the clusters and validating the separation. Another example is a medical setting where we seek groups of patients with similar characteristics. In such applications, doctors seek to understand what makes each cluster unique in terms of demographics, genetic information, and diagnostics.

Reviewer comment (Novelty):
The overall method is just the combination of existing approaches, the novelty is limited.

Author response:
P2- Novelty of technical elements-

We appreciate the feedback regarding the perceived limited novel elements in our framework. We want to address this comment by highlighting the contributions of our work, which goes beyond the technical algorithmic components. 

The novelty of our work could be briefly summarized as follows:
Formulate an interpretable clustering task that highlights sample and cluster-level informative features. 
Generalize the unsupervised feature selection with clustering problem to the dynamic sample-wise setting.
Present an end-to-end neural network-based solution to the proposed interpretable clustering problem. Besides integrating existing methodologies, we have also introduced new (or modified) components. These include (1) a pre-training augmentation scheme that does not require domain-specific knowledge (as used in vision, audio, NLP), see full details Appendix G. (2) The gates coding loss (Eq. 4), which encourages uniqueness of the sample-specific gates. (3) Combining a Gumbel-softmax clustering head with the maxima coding rate loss (eq. 6).
Extensive empirical evaluation of the model's ability to explain its predictions and perform accurate clustering using synthetic and real-world image and tabular data.

As a final remark about novelty, we want to emphasize that many well-celebrated papers in the machine-learning community rely on the combination and adaptation of existing schemes; examples of such works include [1], [2], [3]. We believe that this should be encouraged by the community. The main novelty here is that the presented problem and solution were never addressed.

[1] He et al.” Deep Residual Learning for Image Recognition, 2016” used skip connections (already an existing technique) to enable gradient propagation in deep layers.

[2] Vaswani et al. “Attention Is All You Need”- They used the attention mechanism (commonly applied for text) to image data.

[3] Devlin et al.” BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”- used existing transformer techniques but trained to predict text conditioned on both left and right context in all layers.

Reviewer comment (Theory):
I don not agree with the authors the empirical diversity, faithfulness and uniqueness can represent interpretability.

Author response:
P3 Interpretability metrics-

The main goal of interpretability in ML models is to make the predictions more understandable and transparent for humans; diversity, faithfulness, and uniqueness do not “represent” interpretability; instead, they were proposed in [1,2] as objective metrics to compare different qualities of interpretability models.
Diversity measures how well the interpretability model captures the differences between groups (clusters) and is not overly focused on specific features globally.
Faithfulness evaluates how well the explanations coincide with the mechanism that drives the decision of the ML model. 
Uniqueness generalizes diversity by measuring how the interpretability model can capture different nuances within a cluster. For instance, if a specific cluster has high uniqueness values, the user can interpret that the cluster might contain a union of two (or more) subgroups with distinct characteristics.

We adopt these metrics to provide quantitative comparisons between different schemes. In practice, users can use these metrics to select which method to use and analyze the quality of the clusters and clustering model. We would be happy to include other metrics that the reviewer thinks are more useful for measuring the qualities of our model.

These metrics were proposed in prior work to evaluate model interpretability quality [1,2]. 
We agree with the reviewer that interpretability is a broad concept with different implementations. In this work, we do not focus on the model interpretability in terms of its architecture. This work focuses on predictions interpretability, and we follow previous works [1,2] that proposed diversity and faithfulness. In addition, since we claim that the model produces unique explanations for each sample, we propose the uniqueness metric. The intuition is that the dataset staples are generally not exact duplicates, and each sample has its uniqueness.
	
[1] David Alvarez Melis and Tommi Jaakkola. Towards robust interpretability with self-explaining neural networks. Advances in neural information processing systems, 31, 2018.
[2] Junchen Yang, Ofir Lindenbaum, and Yuval Kluger. Locally sparse neural networks for tabular biomedical data. In the International Conference on Machine Learning, pp. 25123–25153. PMLR, 2022.

Reviewer comment (Writing):
The manuscript was not well prepared. It contains obvious typos, such as the citation "?" in third line of page six.

Author response:
P4 Typos-

Thanks for this comment; we have fixed this issue caused by a technical error. Furthermore, following this comment, we have looked for additional typos. We would be happy to correct any typos the reviewer is aware of.

Reviewer comment (Evaluation):
The improvement in real-world dataset is not significant.

Author response:
P5 Performance improvement- 
While in some datasets, clustering accuracy improvement is not significant compared to SOTA baselines; we argue that those results require careful tuning of the unsupervised feature selection method to obtain these competitive results. In contrast, our method does not require any parameter tuning. Since clustering with tabular data is challenging, several works have shown that NN-based solutions tend to be inferior to unsupervised feature selection models followed by K-means [1] [2]. When we compare our result to K-means,  our method leads to an average improvement of more than 37% in clustering accuracy. Furthermore, the main advantage of our method is interpretability, serving as the only method that provides sample-level and cluster-level informative features while improving the SOTA clustering capabilities on most of the datasets. 

[1] Abrar et al. Effectiveness of Deep Image Embedding Clustering Methods on Tabular Data, 2023.

[2] Solorio-Fernández et al. A review of unsupervised feature selection method, 2019

Reviewer comment (Novelty):
The overall method is just the combination of existing approaches, the novelty is limited.

Author response:
P2- Novelty of technical elements-

We appreciate the feedback regarding the perceived limited novel elements in our framework. We want to address this comment by highlighting the contributions of our work, which goes beyond the technical algorithmic components. 

The novelty of our work could be briefly summarized as follows:
Formulate an interpretable clustering task that highlights sample and cluster-level informative features. 
Generalize the unsupervised feature selection with clustering problem to the dynamic sample-wise setting.
Present an end-to-end neural network-based solution to the proposed interpretable clustering problem. Besides integrating existing methodologies, we have also introduced new (or modified) components. These include (1) a pre-training augmentation scheme that does not require domain-specific knowledge (as used in vision, audio, NLP), see full details Appendix G. (2) The gates coding loss (Eq. 4), which encourages uniqueness of the sample-specific gates. (3) Combining a Gumbel-softmax clustering head with the maxima coding rate loss (eq. 6).
Extensive empirical evaluation of the model's ability to explain its predictions and perform accurate clustering using synthetic and real-world image and tabular data.

As a final remark about novelty, we want to emphasize that many well-celebrated papers in the machine-learning community rely on the combination and adaptation of existing schemes; examples of such works include [1], [2], [3]. We believe that this should be encouraged by the community. The main novelty here is that the presented problem and solution were never addressed.

[1] He et al.” Deep Residual Learning for Image Recognition, 2016” used skip connections (already an existing technique) to enable gradient propagation in deep layers.

[2] Vaswani et al. “Attention Is All You Need”- They used the attention mechanism (commonly applied for text) to image data.

[3] Devlin et al.” BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”- used existing transformer techniques but trained to predict text conditioned on both left and right context in all layers.

Reviewer comment (Theory):
I don not agree with the authors the empirical diversity, faithfulness and uniqueness can represent interpretability.

Author response:
P3 Interpretability metrics-

The main goal of interpretability in ML models is to make the predictions more understandable and transparent for humans; diversity, faithfulness, and uniqueness do not “represent” interpretability; instead, they were proposed in [1,2] as objective metrics to compare different qualities of interpretability models.
Diversity measures how well the interpretability model captures the differences between groups (clusters) and is not overly focused on specific features globally.
Faithfulness evaluates how well the explanations coincide with the mechanism that drives the decision of the ML model. 
Uniqueness generalizes diversity by measuring how the interpretability model can capture different nuances within a cluster. For instance, if a specific cluster has high uniqueness values, the user can interpret that the cluster might contain a union of two (or more) subgroups with distinct characteristics.

We adopt these metrics to provide quantitative comparisons between different schemes. In practice, users can use these metrics to select which method to use and analyze the quality of the clusters and clustering model. We would be happy to include other metrics that the reviewer thinks are more useful for measuring the qualities of our model.

These metrics were proposed in prior work to evaluate model interpretability quality [1,2]. 
We agree with the reviewer that interpretability is a broad concept with different implementations. In this work, we do not focus on the model interpretability in terms of its architecture. This work focuses on predictions interpretability, and we follow previous works [1,2] that proposed diversity and faithfulness. In addition, since we claim that the model produces unique explanations for each sample, we propose the uniqueness metric. The intuition is that the dataset staples are generally not exact duplicates, and each sample has its uniqueness.
	
[1] David Alvarez Melis and Tommi Jaakkola. Towards robust interpretability with self-explaining neural networks. Advances in neural information processing systems, 31, 2018.
[2] Junchen Yang, Ofir Lindenbaum, and Yuval Kluger. Locally sparse neural networks for tabular biomedical data. In the International Conference on Machine Learning, pp. 25123–25153. PMLR, 2022.

Reviewer comment (Novelty):
The paper's novelty appears to be somewhat incremental, as it combines existing unsupervised feature selection (stochastic gates) with deep clustering, lacking significant novel elements.

Author response:
We appreciate the feedback regarding our framework's perceived lack of significant novel elements. We want to address this comment by highlighting the contributions of our work, which goes beyond the technical algorithmic components. 

The novelty of our work could be briefly summarized as follows:
Formulate an interpretable clustering task that highlights sample and cluster-level informative features. 
Generalize the unsupervised feature selection with clustering problem to the dynamic sample-wise setting.
Present an end-to-end neural network-based solution to the proposed interpretable clustering problem. Besides integrating existing methodologies, we have also introduced new (or modified) components. These include (1) a pre-training augmentation scheme that does not require domain-specific knowledge (as used in vision, audio, NLP), see full details Appendix G. (2) The gates coding loss (Eq. 4), which encourages uniqueness of the sample-specific gates. (3) combining a Gumbel-softmax clustering head with the maximal coding rate reduction loss (eq. 6).
Extensive empirical evaluation of the model's ability to explain its predictions and perform accurate clustering using synthetic and real-world image and tabular data.

As a final remark, we want to emphasize that many well-celebrated papers in the community rely on the combination and adaptation of existing schemes; examples include [1], [2], [3]. We believe that this should be encouraged by the community. The main novelty is that the presented problem and solution were never addressed.

[1] He et al.,” Deep Residual Learning for Image Recognition, 2016” used skip connections (already an existing technique) to enable gradient propagation in deep layers.

[2] Vaswani et al. “Attention Is All You Need”- used the attention mechanism (commonly applied to text) for image data.

[3] Devlin et al.” BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”- used existing transformer techniques but trained to predict text conditioned on both left and right context in all layers.

Reviewer comment (Theory):
The main design of the model lacks a theoretical guarantee. For instance, the reasoning behind choosing an autoencoder (AE) over other self-supervised tasks, such as contrastive learning, requires clarification.

Author response:
Indeed, contrastive learning is a powerful tool for self-supervision but requires data augmentations to create “positive pairs.” In vision, audio, and NLP, we can use domain knowledge to design augmentations that preserve each sample's semantic information. Developing such augmentations for tabular data is much more challenging. The main goal of our work is to present a self-supervised scheme for tabular data (for instance, genomic data). Therefore, we decided to use an autoencoder (AE), which was demonstrated effective for clustering and unsupervised feature selection in prior work [4,5,6]. It is important to note that we did not use a standard AE; instead, we have introduced perturbation to the input and latent pairs (see full details Appendix G), which strengthens the ability of our model to identify informative features. We will clarify this in the main text.
Regarding the theoretical analysis, we are currently working on analyzing the feature selection capabilities of our model, but this requires a particular data model and, therefore, does not fit the message of the current paper.
[4] Xie, et al. "Unsupervised deep embedding for clustering analysis" 2016.

[5] Sokar et al. "Where to pay attention in sparse training for feature selection?" 2022.

[6] Li et al. "Reconstruction-based unsupervised feature selection" 2017.

Reviewer comment (Theory):
The method's generalizability to unseen data is not adequately explained. Eq. (6) suggests high computational complexity, necessitating a discussion on the complexity for better understanding.

Author response:
Since our method is fully parametric, it offers generalization capabilities. Our model can predict cluster assignments and informative features for samples not seen during training. We have verified the generalization capabilities of our model in the MNIST experiment. We have predicted the assignments for 10,000 unseen samples. As indicated by the results presented in Table 3, on unseen samples, our model leads to similar clustering accuracies and uses the same amount of selected features. We will clarify this in the text.

Reviewer comment (Experiments):
The experiment comparison seems biased. While the proposed method employs strong feature transformation by DNN, competitors like k-means do not. Hence, a fair comparison with state-of-the-art deep clustering models is essential.

Author response:
Table 4 compares the SOTA NN-based clustering methods on the CIFAR10 dataset. For the real-world tabular data, most deep learning-based clustering schemes typically underperform (see [7]), and unsupervised feature selection schemes followed by K-means usually lead to more accurate cluster assignments (see [8]). It is important to note that CAE (in Table 6) is based on an AE and can also learn complex feature relations using a NN. If the reviewer knows any existing NN scheme for unsupervised feature selection and clustering, we would happily include it as a baseline.
[7] Abrar et al. Effectiveness of Deep Image Embedding Clustering Methods on Tabular Data, 2023.
[8] Solorio-Fernández et al. A review of unsupervised feature selection method, 2019

Reviewer comment (Experiments):
It would be beneficial to discuss the model's performance on a large-scale dataset to provide a comprehensive evaluation.

Author response:
Most of our evaluations focus on datasets where the number of features is larger than the number of samples. In such a setting, noisy features can obscure the clusters, and finding informative features is vital. As demonstrated in Table 6, in these datasets, removing features improves clustering accuracy. We have also evaluated larger datasets such as MNIST and CIFAR-10. Our method could also scale to larger datasets since training is performed in small batches.

Reviewer comment (Theory):
The subscripts in Eq. (6) should be carefully reviewed for accuracy.

Author response:
We will add the following paragraph to clarify this equation:
Given a batch of normalized latent embeddings, we predict for each sample its cluster assignment. Then, for each cluster, we multiply the stacked vectors by their transpose to calculate the correlation matrix, which is also a covariance matrix since the vectors are normalized. Then, we add an identity matrix and calculate the Log determinant of the result. Finally, we sum up the terms from each cluster category k to obtain the loss value. Our goal is to minimize the sum of the volume of each cluster to compress them.