Venue: ICLR 2018 conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposes a neural clustering model following the "Noise as Target" technique. Combining with an reconstruction objective and "delete-and-copy" trick, it is able to cluster the data points into different groups and is shown to give competitive results on different benchmarks.

It is nice that the authors tried to extend the "noise as target" to the clustering problem, and proposed the simple "delete-and-copy" technique to group different data points into clusters. Even tough a little bit ad-hoc, it seems promising based on the experiment results. However, it is unclear to me why it is necessary to have the optimal matching here and why the simple nearest target would not work. After all, the cluster membership is found based on the nearest target in the test stage. 

Also, the authors should provide more detailed description regarding the scheduling of the alpha and lambda values during training, and how sensitive it is to the final clustering performance. The authors cited the no requirement of "a predefined number of clusters" as one of the contributions, but the tuning of alpha seems more concerning.

I like the authors experimented with different benchmarks, but lack of comparisons with existing deep clustering techniques is definitely a weakness. The only baseline comparison provided is the k-means clustering, but the comparisons were somewhat unfair. For all the text datasets, there were no comparisons with k-means on the features learned from the auto-encoders or clusterings learned from similar number of clusters. The comparisons for the Twitter dataset were even based on character-level with word-level. It is more convincing to show the superiority of the proposed method than existing ones on the same ground.

Some other issues regarding quantitative results:
- In Table 1, there are 152 clusters for 10-d latent space after convergence, but there are 61 clusters for 10-d latent space in Table 2 for the same MNIST dataset. Are they based on different alpha and lambda values? 
- Why does NATAC perform much better than NATAC-k? Would NATAC-k need a different number of clusters than the one from NATAC? The number of centroids learned from NATAC may not be good for k-means clustering.
- It seems like the performance of AE-k is increasing with increase of dimensionality of latent space for Fashion-MNIST. Would AE-k beat NATAC with a different dimensionality of latent space and k?
review_title: Interesting work but lack of detailed discussions and thorough quantitative results

final_score: rating: 5: Marginally below acceptance threshold
review_content: This ms presents a new clustering method which combines deep autoencoder and a recent unsupervised representation learning approach (NAT; Bojanowski and Joujin 2017). The proposed method can jointly learn latent features and the cluster assignments. Then the method is tested in several image and text data sets.

I have the following concerns:

1) The paper is not self-contained. The review of NAT is too brief and makes it too hard to understand the remaining of the paper. Because NAT is a fundamental starting point of the work, it will be nice to elaborate the NAT method to be more understandable.

2) Predicting the noise has no guarantee that the data items are better clustered in the latent space. Especially, projecting the data points to a uniform sphere can badly blur the cluster boundaries.

3) How should we set the parameter lambda? Is it data dependent?

4) The experimental results are a bit less satisfactory:
a) It is known that unsupervised clustering methods can achieve 0.97 accuracy for MNIST. See for example [Ref1, Ref2, Ref3].
b) Figure 3 is not satisfactory. Actually t-SNE on raw MNIST pixels is not bad at all. See https://sites.google.com/site/neighborembedding/mnist
c) For 20 Newsgroups dataset, NATAC achieves 0.384 NMI. By contrast, the DCD method in [Ref3] can achieve 0.54.

5) It is not clear how to set the number of clusters. More explanations are appreciated.

[Ref1] Zhirong Yang, Tele Hao, Onur Dikmen, Xi Chen, Erkki Oja. Clustering by Nonnegative Matrix Factorization Using Graph Random Walk. In NIPS 2012.
[Ref2] Xavier Bresson, Thomas Laurent, David Uminsky, James von Brecht. Multiclass Total Variation Clustering. In NIPS 2013.
[Ref3] Zhirong Yang, Jukka Corander and Erkki Oja. Low-Rank Doubly Stochastic Matrix Decomposition for Cluster Analysis. Journal of Machine Learning Research, 17(187): 1-25, 2016.
review_title: interesting method while less satisfactory results

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper presents an algorithm for clustering using DNNs. The algorithm essentially alternates over two steps: a step that trains the DNN to predict random targets, and another step that reassigns the targets based on the overall matching with the DNN outputs. The second step also shrinks the number of targets over time to achieve clustering. Intuitively, the randomness in target may achieve certain regularization effect.

My concerns:
1. There is no analysis on what the regularization effect is. What advantage does the proposed algorithm offer to an user that a more deterministic algorithm cannot?
2. The delete-and-copy step also introduces randomness, and since the algorithm removes targets over time, it is not clear if the algorithm consistently optimizes one objective throughout. Without a consistent objective function, the algorithm seems somewhat heuristic.
3. Due to the randomness from multiple operations, the experiments need to be run multiple times, and see if the output clustering is sensitive to it. If it turns out the algorithm is quite robust to the randomness, it is then an interesting question why this is the case.
4. Does the  Hungarian algorithm used for matching scales to much larger datasets?
5. While the algorithm empirically improve over k-means, I believe at this point combinations of DNN with classical clustering algorithms already exist and comparisons with such stronger baselines are missing. The authors have listed a few related algorithms in the last paragraph on page 1. I think the following one is also relevant:
-- Law et al. Deep spectral clustering learning. ICML 2015.


review_title: overall algorithm is somewhat heuristic

Meta-review: The paper proposes an approach to jointly learning a data clustering and latent representation.  The main selling point is that the number of clusters need not be pre-specified.  However, there are other hyperparameters and it is not clear why trading # clusters for other hyperparameters is a win.  The empirical results are not strong enough to overcome these concerns.
Decision: Reject