Zero-Shot Learning and Clustering for Semantic Utterance Classification
Authors: Yann N. Dauphin, Gokhan Tur, Dilek Hakkani-Tur, Larry Heck
ICLR 2014

Abstract: We propose two novel zero-shot learning methods for semantic utterance classification (SUC) using deep learning. Both approaches rely on learning deep semantic embeddings from a large amount of Query Click Log data obtained from a search engine. Traditional semantic utterance classification systems require large amounts of labelled data, whereas our proposed methods make use of the structure of the task to allow classification without labeled data. We also develop a zero-shot semantic clustering algorithm for extracting discriminative features for supervised semantic utterance classification systems. We demonstrate the effectiveness of the zero-shot semantic learning algorithm on the SUC dataset collected by cite{DCN}. Furthermore, we show that extracting features using zero-shot semantic clustering for a linear SVM reaches state-of-the-art result on that dataset.


Peer review:
[Review]:
I thought this was a very intersting and well written paper. 
Some comments:
I'm ashamed to admit that I didn't what the two methods mentioned were until I read the outro... I thought maybe there would be a fundamentally different approach to both learning AND clustering, or something.  Just to be very clear, I would add this to the 1st sentence of the abstract: '...using deep learning: zero-shot learning and zero-shot clustering'.  I'd also change the 2nd to last sentence of the intro to 'zero-shot learning and zero-shot clustering'.  

While I agree with your choice to distinguish your approach from 'Traditional SUC systems' that 'rely on a large set of labelled examples', I think you should emphasize your use of Query Click Logs more.  I would describe them as analagous with labels in their role in your system (although maybe there is a significant disanalogy I'm overlooking).  This would make things clearer, I think.  

With this in mind (actually, in any case, if I understood this part properly), I think Section 5, paragraph 3, 2nd sentence should say '...using only [...] X, [...] C [...], and Q = query click logs' or something to that effect.  

Also, in Table 1, I believe the first three other ZSL methods you are comparing with do NOT make use of QCLs, and I would emphasize that as well.  I would suggest putting a horizontal dividing line between those three and the next two methods.


some little edits:
6. first paragraph 'However, it is not clear how of a proxy' -> 'However, it is not clear how GOOD of a proxy' (I think that's the word you want?)

next paragraph: 'Is is easy to see THAT' (missing that) 

'If f: X -> Y then we must have...' says the same thing as the next sentence, so I would remove one of them, or at an 'i.e.' or something equivalent.

last sentence 'In general, finding a function satisfying these restrictions is hard,' - this is not technically true, since f itself is such a function and there would generally be other 'trivial' examples.  You should say 'finding a GOOD PROXY function satisfying these restrictions...' or something to that effect.  

next paragraph:
I would rephrase 'We would like these embeddings to cluster...' as 'We would like to cluster these embeddings...' I think this makes it more clear, especially in the context of the next sentence. 

after the equation display:
'The entropy tell us about the uncertainty we have over the class' -> 'The entropy tell us about the uncertainty we have over the CLASSES' (I think this is what you mean.)

last paragraph, you say that P(C | X) is predicted by a DNN, but isn't it predicted by equation (2)?  This uses P(H | X), which IS predicted by a DNN.  A minimal edit I think would be acceptable is changing 'predicted by' to 'predicted using'.  But you could also be more explicit.  


Experiments section 7:
I think you should specify how the ZSL (bag of words) model works; what is it's prediction function?

In that same paragraph, I think the description of the Representative URL Heuristic could be clearer.  I would make these edits:

'We associate each domain with...' -> 'FOR THIS HEURISTIC, we associate each CLASS with...' (I think you mean class, not domain, isn't a domain just a website name?)

next sentence: '...associated with the website for that semantic class' -> '...associated with that class's website'

Next paragraph, 2nd sentence: 'The task is identify...' -> 'The task is TO identify...'