Context-predicting models (more commonly known as embeddings or neural language models) are the new kids on the distributional semantics block. Despite the buzz surrounding these models, the literature is still lacking a systematic comparison of the predictive models with classic, count-vector-based distributional semantic approaches. In this paper, we perform such an extensive evaluation, on a wide range of lexical semantics tasks and across many parameter settings. The results, to our own surprise, show that the buzz is fully justified, as the context-predicting models obtain a thorough and resounding victory against their count-based counterparts.
In an extensive evaluation, Baroni et al. REF ascertain that embeddings of predictive models are superior to their count-based counterparts on word similarity tasks.
Don't count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors
ACL
Computer Science