Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: The authors proposed an improved deep-learning-based representation learning method that provides more efficient features for clustering analysis. 
(1) According to the comparison experiments on several widely used datasets,  the integration of a softmax-formulated orthogonal constraint is able to provide more stable latent feature representation. 
(2) As far as know, the widely-used deep clustering methods used to alternatively optimize the feature representation model parameters and update the anchors that provided by clustering method such as k-means, I am wondering if the proposed method in this study could integrate the two steps in a real end-to-end fashion. 
(3) I was deeply impressed by the far above state-of-the-art values of evaluation metric of this proposed representation learning method. Although the authors provide some distribution illustrations of latent features on CIFAR-10 dataset, what about the visualization on the ImageNet-10? Besides, adding some 'real' visualization results existing in the original image space rather than the latent space could help to illustrate if the proposed method could mine visually meaningful concepts from the view of visual contents. 
review_title: A good submission that aims at a valuable and fundamental machine learning task that shows some improvement.

final_score: rating: 7: Good paper, accept
review_content: This paper proposes a clustering-friendly representation learning method using instance discrimination and feature decorrelation. Instance discrimination loss and feature decorrelation loss are combined to optimize the network. The paper is well qritten and experimental results are good. I have some questions about this paper:
1. There is no ablation analysis about the two loss terms in Eq.(6). What about the contributions of the two loss terms? 
2. What is the motivation of Eq.(3)? I.e., why the "=" holds between the second and third expressions?
review_title: Paper 1734 review

final_score: rating: 6: Marginally above acceptance threshold
review_content: One of the main contributions is this idea of feature decorrelation where they encourage the representation features to be independent / orthogonal.
The other is instance discrimination. This aims to capture the similarity between individual data points. Both of these are interesting contributions to the field of 'deep clustering'.


Besides the stated contributions, I thought there were a number of other positive aspects of this. 
	A) I thought that the spectral clustering connection was nice and I am glad the authors included it.
	B) The evaluation is fairly detailed. I particularly appreciate the fact that the authors used datasets that are somewhat larger than often used in the literature (MNIST and CIFAR-10 vs CIFAR-100 and ImageNet-10). The inclusion of the study of the temperature parameter also helped clarify a few questions I had when reading it. 
	C) Finally, the evaluation clearly shows the benefit of their contributions in terms of performance. 
	
There are a number of questions I have with the work as is.
	A) Given the two methods proposed,  IDFO, IDFD, neither of which outperforms the other on all tasks, and given this is unsupervised learning, how does one know which method to use?
	B) Why was the alpha parameter set to 1 for IDFD? How does one know what to set this to for different datasets? If it's always 1, why is it included at all?  This is particularly important to understand in unsupervised settings. 
	C) The impact of data augmentation is discussed in the supplementary but this is stated as being extremely important to the performance of the model. It is unclear to me whether the results in the main text include the augmentation process?  If so, then given this, I think it should be stated in the main text as it has an effect on both instance discrimination and feature decorrelation considering the addition of augmented images.  The results in supplementary Table 4 include KNN and don't match up with the main results in the main text which further confused me.
	D) I was left wondering how well this method works on non-image data? Other works in the literature have explored this. 
	E) For Fig. 2 is this ACC calculated on the validation set or test set?
	F) What were the effects of resizing the ImageNet images? Can this model handle larger images, and if so, how does this effect performance?

Minor
	A) References are badly formatted in Table 3.
	

Overall, my questions above notwithstanding, I think this is an interesting contribution which shows the benefit of instance discrimination and feature decorrelation for deep clustering.
review_title: This paper proposes two interesting contributions to the 'deep clustering' literature and demonstrates the benefit experimentally.

Meta-review: This paper received mostly positive reviews. The reviewers praised the strong performance when compared with previous work.
Also, the evaluation clearly shows the benefit of the proposed contributions in terms of performance.
Most concerns raised by reviewers were properly addressed in the rebuttal.

Lack of comparison to several previous works has been noted in a comment, but the authors clarified this concern, stating that the current work is a “large deviation from prior works”. The authors promised to include the missing references into the comparison.

Given the reviews, comments, and author's answers, I suggest acceptance.
Decision: Accept (Poster)