Task: Language Identification
Input (Spanish, English): This paper proposes a method for evaluating topic quality based on using word embeddings to calculate similarity (either directly or indirectly via matrix factorisation), achieving impressive results over standard datasets.  The proposed method represents a natural but important next step in the evolutionary path of research on topic evaluation. The thing that troubled me most with the results was that, while you achieve state-of-the-art results for all three datasets, there are large inconsistencies in which methods perform and which methods perform less well (below the state of the art). In practice, none of the proposed methods consistently beats the state of the art, and the SVD-based methods perform notably badly over the genomics dataset. For someone who wants to take your method off the shelf and use it over any arbitrary dataset, this is a considerable worry. I suspect that the lower results for SVD over genomics relate to the proportion of OOV terms (see comment below), and that it may be possible to automatically predict which method will perform best based on vocab match with GloVe etc., but there is no such discussion in the paper.  Other issues:  - the proposed method has strong similarities with methods proposed in the   lexical chaining literature, which I would encourage the authors to read up   on and include in any future version of the paper  - you emphasis that your method has no parameters, but the word embedding   methods have a large number of parameters, which are implicit in your   method. Not a huge deal, but worth acknowledging  - how does your method deal with OOV terms, e.g. in the genomics dataset   (i.e. terms not present in the pretrained GloVe embeddings)? Are they simply   ignored? What impact does this have on the method?  Low-level issues:  - in your description of word embeddings in Section 2.1, you implicitly assume   that the length of the vector is unimportant (in saying that cosine   similarity can be used to measure the similarity between two vectors). If   the vectors are unit length, this is unproblematic, but word2vec actually   doesn't return unit-length vectors (the pre-trained vectors have been   normalised post hoc, and if you run word2vec yourself, the vector length is   certainly not uniform). A small detail, but important.  - the graphs in Figure 1 are too small to be readable
Output (English): en