Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper presents a method where they 1) use an ensemble of networks to cluster unlabeled data and assign pairs of data points a cluster label only if all networks agree that the pair belongs to a cluster 2) use the labeled pairs to create a similarity matrix and find a "tight" cluster or set of points that are all very similar to each other. The paper then uses the "labelled" points for semi-supervised learning with a proposed ensemble of models. 

The paper's method of creating high precision labels using their multi-step clustering algorithm with information measures is quite interesting. The experiment results look promising. 
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: This paper proposed an unsupervised learning method of clustering using semi-supervised clustering as a bridge. The method first trains an ensemble of clustering models and use the edge-level majority vote to determine a graph, and then applies rule to get partial clustering signals to feed the final semi-supervised clustering. The scheme is in an iterative fashion to further enhance the quality. I find this paper interesting and somewhat novel, with the following comments.

1. In algorithm 1, is it possible that too many nodes are removed so one cannot get k clusters in the end? Though finding cliques are time consuming, have the authors conducted experiments to see the difference between the real clique finding algorithm and the greedy one proposed?

2. Does the ensemble clustering step have stability issue regarding the method used? If a different clustering method is used, will the graph constructed later change drastically?

3. The writing. First line of section 3, figure 4 seems to point to figure 1. Section 2 seems to have format issue at the beginning. Section 5 could be merged with section 2.
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: This paper proposes a method for unsupervised clustering. Similarly to others unsupervised learning (UL) papers like "Deep Clustering for Unsupervised Learning of Visual Features" by Caron et al., they propose an algorithm alternating between a labelling phase and a training phase. Though, it has interesting differences. For example, unlike the Caron et al. paper, not all the samples get assigned a labels but only the most confident ones. These samples are determined by the pruning of a graph whose edges are determined by the votes of an ensemble of clustering models. Then, these pseudo labels are used within a supervised loss which act as a regularizer for the retraining of the clustering models.

Novelties /contributions/good points:
* Votes from the clustering models to create a graph
* Using a graph to identify the most important samples for pseudo labelling
* Modification of the ladder network to be used as clustering algorithm
* Good amount of experiments and good results

Weaknesses:
* The whole experiment leading to Table 1 in page 2 is unclear for me. I have trouble understanding the experiment settings. Could you please rephrase it. About initial/ final clustering for example and the rest as well. The whole thing puzzles me whereas the experiments section at the end is much more clear.
* Lack of motivation about why using the Ladder method rather than another one. Other recent methods have better results in semi-supervised learning.
* Algorithm 1 seems quite ad-hoc. Do more principled algos exist to solve this problem ? You could write about it and at least explain why it would not be feasible here. The sentence "The intuition is that most of the neighbours of that node will also be connected with each other" is unmotivated: no empirical proof for this ?
* Related work section is too light. It is an important section and should really not be hidden or neglected.
* In the experiments, you could add the "Deep Clustering for Unsupervised Learning of Visual Features"  as baseline as well even if they use it for unsupervised learning as they do clustering as well.
* In the experiments, you use the features extracted from ResNet-50 but what about finetuning this network rather than adding something on top or even better starting from scratch. Because here CIFAR-10 benefits greatly from the ImageNet features. I know that you should reproduce the settings from other papers but it might be good to go a bit beyond. Especially, if the settings of previous papers are a bit faulty. 
* Regarding, the impact of number of models in section D of the appendix, there is no saturation at 10 models. So how many models are necessary for saturation of the performance ?
* Minor point: several times, you write "psuedo".

Conclusion: the algorithm is novel and represents a nice contribution. Though, there are a lot of weaknesses that could be solved. So, I am putting "Weak accept" for the moment but it could change towards a negative rating depending on the rebuttal.

review_title: Official Blind Review #3

Meta-review: The authors addressed the issues raised by the reviewers, so I suggest the acceptance of this paper.
Decision: Accept (Poster)