Cross-lingual Short-text Entity Linking: Generating Features for Neuro-Symbolic Methods (2022)

Abstract: Entity linking (EL) on short text is crucial for a variety of industrial applications. Compared with general long-text EL, short-text EL poses particular challenges as the limited context restricts the clues one can leverage to disambiguate textual mentions. On the other hand, existing studies mostly focus on black-box neural methods and thus lack interpretability, which is critical to industrial applications in certain areas. In this study, we extend upon LNN-EL, a monolingual short-text EL method based on interpretable first-order logic, by incorporating three sets of multilingual features to enable disambiguating mentions written in languages other than English. More specifically, we use multilingual autoencoding language models (i.e., mBERT) to capture the similarities between the mention with its context and the candidate entity; we use multilingual sequence-to-sequence language models (i.e., mBART and mT5) to represent the likelihood of the text given the candidate entity. We also propose a word-level context feature to capture the semantic evidence of the co-occurring mentions. We evaluate the proposed xLNN-EL approach on the QALD-9-multilingual dataset and demonstrate the cross-linguality of the model and the effectiveness of the features.

## introduction entity linking (el), also known as entity disambiguation, is the task of linking textual mentions appearing in a document to the corresponding entities associated with a target knowledge base like dbpedia @xcite . as a fundamental task in natural language processing and information extraction, entity linking is crucial for a variety of applications such as semantic search, recommendation systems and chatbots @xcite . historically, relevant studies of entity linking mostly focus on long text scenario (i.e., documents) @xcite @xcite @xcite . typically, such approaches mainly depend on specifically-designed features of candidates (e.g., priors), local context compatibility, and global coherence across the document @xcite . with the rapidly growing short text on the web, e.g., search queries, social media posts, news headlines, etc., short-text entity linking has attracted increasing attention from researchers due to its potential for various industrial applications. however, long-text entity linking methods barely maintain the same level of performance on short text, as they heavily rely on document-level global coherence, i.e., the idea of collective entity linking @xcite , and short text (e.g., a single sentence or search query) cannot provide rich context and global signals for disambiguation @xcite . to tackle the task of short-text entity linking, different methods are being proposed to exploit the short and limited context @xcite @xcite . for example, chen et al. @xcite try to map the sparse short text to a topic space such that a topic coherence can be achieved through a specifically designed optimization objective, e.g., they aim to infer the topic is literature against movie from the word read in "read harry potter". gu et al. @xcite try to enhance the interactions between the local context and the candidate entity in multiturn multiple-choice manner, so that global disambiguation can finally be achieved. more recently, autoregressive entity linking models, e.g., genre @xcite and mgenre @xcite demonstrate superior performance and are gaining a lot of attention. with the power of large pre-trained sequence-to-sequence language models @xcite , genre and mgenre are able to directly generate unique entity names conditioned on the context, yielding state-of-the-art results on multiple datasets. recently, jiang et al. propose lnn-el, the first neuro-symbolic short-text entity linking method that combines first-order logic (fol) rules with neural learning, and shows competitive performance against deep-learning-based black-box methods @xcite . essentially, lnn-el uses fol rules as a glue to combine different features into a final linking model, and demonstrates interpretability due to the interpretable nature of fol. nonetheless, lnn-el is a monolingual model that only supports english, and does not meet the demands of modern industrial applications where enterprises need to establish effective cross-language interactions with users from all over the world. to enable this approach to facilitate rapidly growing global business, in this study, we extend upon lnn-el by incorporating three sets multilingual features to make it cross-lingual, i.e., xlnn-el. first, we propose a word-level cross-lingual context feature that aims to capture the semantic evidence of co-occurring mentions. second, we use multilingual autoencoding language models (i.e., mbert @xcite ) to capture the similarities between the mention with its context and the candidate entity with a four-way feature. third, we use multilingual sequence-to-sequence language models (i.e., mbart @xcite and mt5 @xcite ) to represent the likelihood of the text given the candidate entity. more specifically, we try to reconstruct the text conditioned on the candidate's description, based on a fine-tuned generative language model, and it results in another two-way feature indicating the likelihood of the text. we evaluate xlnn-el on the qald-9-multilingual dataset with the state-of-theart black-box method mgenre @xcite , and the results demonstrate the effectiveness of the proposed features. our contribution can thus be summarized as follows: ## xlnn-el we propose xlnn-el, a cross-language extension of lnn-el, where we seek to facilitate the model with better cross-linguality by incorporating a set of new features allowing it to link non-english mentions to the english knowledge base. following lnn-el, we take the english dbpedia as the target knowledge base @xcite . as to candidate retrieval, we adopt a hybrid method of pivotscr @xcite and mgenre @xcite . essentially, for each mention m i , we take the union of their outputs and generate a set of |c i | = 250 candidate entities, roughly reaching a 95% recall rate for each language foot_0 . formally, given a single sentence or search query t containing a set of mentions @xmath0, m 2 , . . . , m p }, a triple m i , c i , l i is generated for each mention m i . l i is a list of binary labels for the mention-candidate pair (m i , e ij ) where e @xmath1. the entity name and textual description of e ij is denoted by e ij .name and e ij .desc, respectively. for each candidate e ij , a set of predefined features f w (m i , e ij ) is generated. in this section, we introduce the three cross-language features f @xmath2. ## conclusion in this study, we extend upon lnn-el by incorporating three sets of multilingual features to enable disambiguating mentions written in languages other than english. this study also indicates di-rections of future work, as the results demonstrate language-specific patterns for the features and rules.