Learning a high-dimensional dense representation for vocabulary terms, also known as a word embedding, has recently a racted much a ention in natural language processing and information retrieval tasks. e embedding vectors are typically learned based on term proximity in a large corpus. is means that the objective in well-known word embedding algorithms, e.g., word2vec, is to accurately predict adjacent word(s) for a given word or context. However, this objective is not necessarily equivalent to the goal of many information retrieval (IR) tasks. e primary objective in various IR tasks is to capture relevance instead of term proximity, syntactic, or even semantic similarity. is is the motivation for developing unsupervised relevance-based word embedding models that learn word representations based on query-document relevance information. In this paper, we propose two learning models with di erent objective functions; one learns a relevance distribution over the vocabulary set for each query, and the other classi es each term as belonging to the relevant or non-relevant class for each query. To train our models, we used over six million unique queries and the top ranked documents retrieved in response to each query, which are assumed to be relevant to the query. We extrinsically evaluate our learned word representation models using two IR tasks: query expansion and query classi cation. Both query expansion experiments on four TREC collections and query classi cation experiments on the KDD Cup 2005 dataset suggest that the relevance-based word embedding models signi cantly outperform state-of-the-art proximity-based embedding models, such as word2vec and GloVe. Word representation, neural network, embedding vector, query expansion, query classi cation ACM Reference format:
Zamani and Cro REF recently proposed relevance-based word embedding models for learning word representations based on the objectives that ma er for IR applications.
Relevance-based Word Embedding
SIGIR '17
Computer Science