IR like a SIR : S ense-enhanced I nformation R etrieval for M ultiple L anguages (2021)

Abstract: With the advent of contextualized embeddings, attention towards neural ranking approaches for Information Retrieval increased considerably. However, two aspects have remained largely neglected: i) queries usually consist of few keywords only, which increases ambiguity and makes their contextualization harder, and ii) performing neural ranking on non-English documents is still cumbersome due to shortage of labeled datasets. In this paper we present SIR (Sense-enhanced Information Retrieval) to mitigate both problems by leveraging word sense information. At the core of our approach lies a novel multilingual query expansion mechanism based on Word Sense Disambiguation that provides sense definitions as additional semantic information for the query. Importantly, we use senses as a bridge across languages, thus allowing our model to perform considerably better than its supervised and unsupervised alternatives across French, German, Italian and Spanish languages on several CLEF benchmarks, while being trained on English Robust04 data only. We release SIR at https://github.com/SapienzaNLP/sir .

## introduction information retrieval (ir) is the task of retrieving from a large collection of unstructured information -generally textual documents -those items deemed relevant to users, and which are expressed by a query -typically a few keywords. ir systems have become an integral part of our daily lives, as web search engines testify, by allowing us to address distinct search tasks. relevance is the key notion in ir: indeed, the core component of an ir system is the ranking module, which estimates the relevance of a document to a given query. this is achieved through a ranking function that complies with an underlying formal modeling such as the vector space model, probabilistic models and, more recently, neural models @xcite . lately, ir systems have begun taking advantage of these latter models, whose aim is learning continuous representations capable of grasping the semantics of the text, as opposed to the traditional lexical approaches comprising the bag-of-words representation. in this new line of research, following the success of neural models in several natural language processing (nlp) tasks, researchers employed contextualized word representations @xcite in ir to capture semantic aspects of texts (for query and documents) which prove beneficial to ranking approaches @xcite . moreover, thanks to the unsupervised training strategy of contextualized language models, i.e., masked language modeling, it is feasible to train multilingual models which are able to encode sentences across languages within the same semantic space. nonetheless, there are challenges peculiar to ir that may hinder the effectiveness of contextualized embeddings. for example, queries are typically composed of just a few keywords, which may not be sufficient to assess the relevance of documents to a query effectively. in classical ir, the technique of query expansion is employed to provide more context about users' actual needs @xcite , by exploiting synonymous terms to overcome the vocabulary mismatch problem. however, this is not suitable for neural language models which are trained to process well-formed sentences. this issue is even more pronounced when dealing with languages other than english, where the lack of training data hinders the use of machine learning in the multilingual setting. recently, word sense disambiguation (wsd) has received greatly increased attention @xcite , reporting large improvements not only in english @xcite @xcite @xcite , but also across other languages @xcite . we argue that word senses, thanks to their glosses, i.e., sentences defining word meanings, can provide valuable information to enrich the input query and to aid retrieving relevant documents that are semantically related. moreover, multilingual sense vocabularies (where concepts are lexicalized with synonymous words in different languages) may provide a bridge across languages, leading neural models to perform better in a zeroshot setting. based on these hypotheses, this paper makes the following contributions: our findings show that word definitions are indeed beneficial to the task, allowing sir to better contextualize queries and thus match more relevant documents in respect of all its baselines. ## related work information retrieval approaches have long relied on simple statistical metrics based on term frequency, such as tf-idf and bm25 @xcite , to represent texts and to match documents against a given query. these methods are still used as strong baselines nowadays @xcite , especially because they perform retrieval in an unsupervised and efficient way. in the last decade, two different kinds of neural approaches to ir have been defined @xcite : the first aims at encoding queries and documents within the same vector space @xcite ; the second, instead, focuses on learning an estimator for the relevance of a document with respect to a query @xcite . more recently, with the advent of transformer-based language models such as bert @xcite , contextualized representations rapidly got incorporated into retrieval models (macavaney et al., 2019) -which previously had relied on static embeddings only -mainly by pairing contextualized models with a binary classifier to compute a score per query-document @xcite or query-sentence pair @xcite . nevertheless, most of the supervised works focused on the english retrieval task, where enough labeled data are available to train a neural model. instead, while datasets in languages other than english do exist in several tracks of trec @xcite or clef @xcite , they are rather small and not suitable for training deep neural networks. in this setting, multilingual pretrained language models came out as an effective solution, and showed themselves able to successfully leverage annotations in one language (typically english) and perform retrieval in other languages, e.g., @xcite or chinese, arabic, french, hindi and bengali @xcite . however, by relying on large pretrained language models, these approaches assume that queries are expressive enough to model their underlying semantics, which is not always the case. this is a long-standing issue in ir, and one which has stimulated extensive research for years. different approaches to query expansion such as markov chains @xcite , term classification @xcite , and static word embeddings @xcite have been applied effectively to improve query representation. more recently, researchers have tried to tackle the problem from the opposite perspective by expanding documents @xcite @xcite or single passages (macavaney et al., 2020a) with relevant queries. nevertheless, most approaches to query expansion do not cope well with the issue of word ambiguity, which makes it hard to model and represent words when not enough context is provided. g 1 -having more than one spouse at a time g 2 -having more than one husband at a time g 3 -having more than one wife at a time q 1 q @xmath0, polyandry, polygyny q 2 q3 exact search for l2 a r e s query encoder a look at the roots and prevalence of polygamy in the world today. q @xmath1: [g 1 , "/", g 2 , "/", g 3 , ":", q] (a) query expander (b) document ranker [cls] q e [sep] d-[sep] [cls] q e [sep] d+ [sep] transformer encoder e [cls] e qe e [sep] e d-e [sep] e [cls] e qe e [sep] e d+ e [sep] linear pairwise softmax sense embeddings b e r t s 1 s 2 s|kb| ... @xmath2... ??? word sense disambiguation (wsd) is specifically tailored to resolve this issue, and several attempts were made in the past to include word senses within ir pipelines. these early attempts, unfortunately, did not produce encouraging results @xcite @xcite . indeed, @xcite emphasised that the effectiveness of wsd integration was diminished by the inaccuracies in disambiguation. a little over a decade later, instead, zhong and ng (2012) presented a successful application of wsd in ir by incorporating word senses and synonym relations into a language modeling approach. in addition, further developments over the years led to the remarkable performance attained by modern wsd models, which now perform close to the inter-annotator agreement upper bound @xcite @xcite . this makes us optimistic that these models are finally suitable to be used within downstream tasks. differently from previous works, in this paper, we explore this possibility and focus on enriching the query context by devising a neural approach to first retrieve word senses for the input query terms, and then encode their definitions together with query and documents to perform end-to-end document ranking. to the best of our knowledge, this is the first time a word sense disambiguation approach has been employed to expand the query with sense definitions and we show that this is not only beneficial in the monolingual setting but also cross-lingual zero-shot settings. ## preliminaries in this section, we describe the task we are tackling and the resources we exploit. ## experiments in this section, we describe the baselines we compare our approach with, as well as the tasks and datasets used for training and evaluating them. ## conclusions in this paper we presented sir, a novel approach for ranking documents in multiple languages. our approach is the first to take advantage of a wsd model to expand the input query with sense definitions as additional semantic information. by evaluating sir on multiple gold information retrieval benchmarks across languages, we show that our approach consistently improves over its main competitors that do not have access to sense glosses, thus demonstrating that such information is beneficial for the english retrieval task, as well as in the zero-shot cross-lingual setting. in addition, through a simple qualitative analysis, we highlight the advantages and disadvantages of sir, suggesting promising directions for better utilizing wsd to improve ir models. we release sir at https://github.com/sapienzanlp/sir to ease future research in this direction.