Entity Retrieval from Multilingual Knowledge Graphs (2022)

Abstract: Knowledge Graphs (KGs) are structured databases that capture real-world entities and their relationships. The task of entity retrieval from a KG aims at retrieving a ranked list of entities relevant to a given user query. While English-only entity retrieval has attracted considerable attention, user queries, as well as the information contained in the KG, may be represented in multiple—and possibly distinct—languages. Furthermore, KG content may vary between languages due to different information sources and points of view. Recent advances in language representation have enabled natural ways of bridging gaps between languages. In this paper, we therefore propose to utilise language models (LMs) and diverse entity representations to enable truly multilingual entity retrieval. We propose two approaches: (i) an array of monolingual retrievers and (ii) a single multilingual retriever, trained using queries and documents in multiple languages. We show that while our approach is on par with the significantly more complex state-of-the-art method for the English task, it can be successfully applied to virtually any language with a LM. Furthermore, it allows languages to benefit from one another, yielding significantly better performance, both for low- and high-resource languages.

## introduction knowledge graphs (kgs) are key for many search applications. consider, for example, the user query "chess world champions". modern search engines often present users with a list of world chess champions along with additional facts encoded as relations in a kg. the queries themselves, as well as the information contained in a kg, may be represented in multiple-and possibly distinctlanguages. this poses a challenge to traditional entity retrieval methods usually optimised for a single language. in this work, we aim to tackle the task of multilingual entity retrieval: given a query in any language, and a kg holding data in multiple languages, retrieve a ranked list of relevant entities. the task of entity retrieval, when both the query and kg are in english, is well-studied. recent years have seen remarkable progress, resulting in over 20% improvement on dbpedia-entity v2 (de-v2), the standard test collection for the task @xcite . works like esim @xcite and kewer @xcite utilised word embedding techniques to represent entities and user queries in the same latent space. meanwhile, em-bert @xcite combines a powerful entity extractor that enhances user queries with a pre-trained language model (lm), fine-tuned on another ranking task, to establish a new state-of-the-art. these methods, however, operated on a single language at a time and were not studied in a multilingual setting. while de-v2 is an english-only collection, wikipedia and dbpedia @xcite provide a unique opportunity: because the contributors to each language edition come from different backgrounds and have different views, we often see rich and diverse entity representations that go well beyond word-for-word translation. moreover, many entities are available only in some chapters but not in others (see appendix g for examples). thanks to its graph-based nature, dbpedia facilitates mapping between languages and different entities representing the same subject. this, in turn, allows us to build rich, multilingual representations. expanding de-v2 to multiple languages, however, carries several risks. the collection was developed based on english dbpedia; therefore, its pooling stage uses keyword-based retrievers optimised for english. moreover, annotators were only presented with english content. in this paper, we discuss these challenges through example queries 1 and stress the importance of building a truly multilingual collection end-to-end. to address the task of multilingual entity retrieval, we introduce berte, a multi-and crosslingual entity ranking framework. despite its simple design and its flexibility to use any lm out of the box, it is comparable to the state-of-the-art on de-v2 in its original english form and thrives in a variety of languages, including spanish, arabic, and hebrew. furthermore, our experiments show that berte can benefit greatly from combining information from multiple languages to boost its performance, establishing a new state-of-the-art for a large subset of the queries. the main contributions of our work are threefold: (i) a novel and simple yet effective entity retriever for the monolingual setup; (ii) a system for multilingual entity retrieval; and (iii) a systematic way to extend de-v2 to multiple languages accompanied with a set of strong baseline results. ## multilingual entity retrieval to tackle the entity retrieval task, we follow a standard two-staged approach: we first use a keywordbased method to retrieve a set of entities and then rerank them. both steps rely exclusively on textual information extracted from the kg. similar to the guidelines in de-v2, each entity representation is composed by concatenating its direct literal attributes. 1 for an entity e, with n a textual attributes, its representation e d is defined as [a t , a la , a 1 , . . . , a na ], where a t is the title, a la is the long abstract and a i is the i th attribute. appendix a provides an overview of our proposed system. for the first-stage retrieval, we use the wellestablished and language-agnostic bm25. it scores documents in relation to a query based on term frequency, document frequency, document length and term saturation. where possible, and to allow a fair 1 unlike de-v2, we use flat, unfielded documents. ## empirical evaluation we conducted a series of experiments on de-v2 to analyse our proposed approaches. we also used de-v2's 5-fold train-test split to allow comparison with previous works. β, the weight given to the first-stage retriever, is fine-tuned using a validation set (one training fold). we found β = 0.75 to work best for english and used it across all experiments. in each language l, we adopt the same procedure when training the respective berte l model. for every training query q and relevant entity e + , we generate 10 triplets of the form q, e + , e -, where e -is a randomly drawn judged non-relevant entity for q. we use a pairwise softmax cross-entropy loss, adamw optimiser, with a learning rate of 1e - foot_3 , and train for 20,000 steps, with a batch size of 32. the embedding vectors are of size 128. we first evaluate berte on the original english collection and the arabic subset of de-v2, the only publicly available non-english resource for the task. we then discuss how to extend de-v2 to other languages systematically and evaluate berte l (monolingual lms) and multiberte (multilingual lms) on the complete set of queries, machine-translated to spanish, arabic, and hebrew. finally, we demonstrate how english can benefit from other languages. note that we optimise for english only and fix β = 0.75 for all experiments and languages. optimising β per language will likely further improve results. ## conclusion in this paper, we introduced berte, a highly effective multilingual entity retrieval system. we showed that in a monolingual environment, it is on par with current state-of-the-art methods on de-v2 despite being simpler and requiring less data. we then explored the multilingual setup, where both the graph and the queries may be presented in multiple languages. we proposed a systematic way to extend de-v2 beyond english and discussed the risks of such approach. we believe it is vital for the community to curate truly multilingual collections that come from different sources and involve native speakers. to address the multilingual retrieval task, we considered both a collection of monolingual models and a single multilingual one. we showed that combining the scores from different languages significantly boosts the performance of low and high-resourced languages. our work can enable many downstream tasks. consider, for example, a virtual assistant answering questions in arabic about a topic covered mainly in the english edition of wikipedia or an english speaker analyst covering a multi-national company interested in taking diverse points of view coming from content in different languages. in both cases, berte allows handling queries and kgs in multiple languages. we hope that this work opens interesting avenues of research. as discussed in appendix c, the improvements brought by berte are orthogonal to those by the other state-of-the-art method, em-bert. therefore, we hope that combining each method's contributions will establish a new stateof-the-art for the english task. on the multilingual front, works that adapt the retrieval task to user preference, such as language, region, or past actions, may benefit from the flexibility of berte in combining different sources.