Venue: ICLR 2017 workshop

Reviews:

final_score: rating: 7: Good paper, accept
review_content: This paper introduces an auxiliary cost function which forces the representation learnt in a particular layer to be useful for clustering. This can be added to any classification model or unsupervised model like autoencoder. Authors show that adding this loss helps in better clustering of examples in a binary classification task and auto-encoder task. The model does not have access to any cluster information and it learns to group examples based on their characteristics.

The proposed objective function roughly maximizes the KL-Divergence between the probability distributions induced from the row vectors in a weight matrix. I like the idea of directly considering weights instead of considering the unit activations as done in DeCov. Also from experiments, proposed approach does better than DeCov.

While this is an interesting idea, I encourage the authors to verify the benefits of the proposed loss in a variety of datasets. Also it looks like one can combine both DeCov and the proposed loss. They look complementary.

Can you also plot Figure 2 with DeCov loss?

Pros:
+ New cost function for learning representations useful for clustering
+ Proof of concept experiments that show the method works.

Cons:
- Needs benchmarking with several tasks and datasets.

review_title: A new auxiliary cost function to learn representations useful for clustering.

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper proposes a regularizer to promote representations that 
can easily be clustered in the context of time series classification.

The regularizer proposed by the authors looks at the pairwise KL between 
the softmax-induced probability distribution from each row of the weight matrix. 
The rationale is that a weight matrix with rows that are "different" enough 
from each other would yield more diversity in the representation. 

While favorable experiments are provided, I am not convinced in the reasoning underlying 
the proposed cost function. I don't see how weight matrices with "different rows"
would yield better representations. There is a complex interaction with other layers, 
learning algorithm, nonlinearities and hidden states which could easily destroy 
the intended benefits of this regularizer. It's also not clear why looking at the 
rows in probabilistic sense (through the softmax and KL) is necessary at all. 
Why not simply taking the L2 for example ? 


review_title: Insufficient motivation for the proposed solution

Decision: Accept