Task: Language Identification
Input (Spanish, English): This paper proposes integrating word sense inventories into existing approaches for the lexical substitution task by using these inventories to filter candidates. To do so, the authors first propose a metric to measure the mutual substitutability of sense inventories with human judgments for the lexsub task, and empirically measure the substitutability of inventories from various sources such as WordNet and PPDB. Next, they propose clustering different paraphrases of a word from PPDB using a multi-view clustering approach, to automatically generate a sense inventory instead of using the aforementioned inventories. Finally, they use these clusters with a naive (majority in top 5) WSD technique to filter existing ranked list of substitution candidates.  - Strengths:  * The key idea of marrying vector space model based approaches and sense inventories for the lexsub task is useful since these two techniques seem to have complementary information, especially since the vector space models are typically unaware of sense and polysemy.  * The oracle evaluation is interesting as it gives a clear indication of how much gain can one expect in the best case, and while there is still a large gap between the oracle and actual scores, we can still argue for the usefulness of the proposed approach due to the large difference between the unfiltered GAP and the oracle GAP.  - Weaknesses:  * I don't understand effectiveness of the multi-view clustering approach. Almost all across the board, the paraphrase similarity view does significantly better than other views and their combination. What, then, do we learn about the usefulness of the other views? There is one empirical example of how the different views help in clustering paraphrases of the word 'slip', but there is no further analysis about how the different clustering techniques differ, except on the task directly. Without a more detailed analysis of differences and similarities between these views, it is hard to draw solid conclusions about the different views.                                    * The paper is not fully clear on a first read. Specifically, it is not immediately clear how the sections connect to each other, reading more like disjoint pieces of work. For instance, I did not understand the connections between section 2.1 and section 4.3, so adding forward/backward pointer references to sections should be useful in clearing up things. Relatedly, the multi-view clustering section (3.1) needs editing, since the subsections seem to be out of order, and citations seem to be missing (lines 392 and 393).  * The relatively poor performance on nouns makes me uneasy. While I can expect TWSI to do really well due to its nature, the fact that the oracle GAP for PPDBClus is higher than most clustering approaches is disconcerting, and I would like to understand the gap better. This also directly contradicts the claim that the clustering approach is generalizable to all parts of speech (124-126), since the performance clearly isn't uniform.  - General Discussion:  The paper is mostly straightforward in terms of techniques used and experiments. Even then, the authors show clear gains on the lexsub task by their two-pronged approach, with potentially more to be gained by using stronger WSD algorithms.  Some additional questions for the authors :  * Lines 221-222 : Why do you add hypernyms/hyponyms? * Lines 367-368 : Why does X^{P} need to be symmetric? * Lines 387-389 : The weighting scheme seems kind of arbitrary. Was this indeed arbitrary or is this a principled choice? * Is the high performance of SubstClus^{P} ascribable to the fact that the number of clusters was tuned based on this view? Would tuning the number of clusters based on other matrices affect the results and the conclusions? * What other related tasks could this approach possibly generalize to? Or is it only specific to lexsub?
Output (English): en