Linking Entities to Unseen Knowledge Bases with Arbitrary Schemas (2021)

Abstract: In entity linking, mentions of named entities in raw text are disambiguated against a knowledge base (KB). This work focuses on linking to unseen KBs that do not have training data and whose schema is unknown during training. Our approach relies on methods to flexibly convert entities with several attribute-value pairs from arbitrary KBs into flat strings, which we use in conjunction with state-of-the-art models for zero-shot linking. We further improve the generalization of our model using two regularization schemes based on shuffling of entity attributes and handling of unseen attributes. Experiments on English datasets where models are trained on the CoNLL dataset, and tested on the TAC-KBP 2010 dataset show that our models are 12% (absolute) more accurate than baseline models that simply flatten entities from the target KB. Unlike prior work, our approach also allows for seamlessly combining multiple training datasets. We test this ability by adding both a completely different dataset (Wikia), as well as increasing amount of training data from the TAC-KBP 2010 training set. Our models are more accurate across the board compared to baselines.

## introduction entity linking consists of linking mentions of entities found in text against canonical entities found in a target knowledge base (kb). early work in this area was motivated by the availability of large kbs with millions of entities @xcite . most subsequent work has followed this tradition of linking to a handful of large, publicly available kbs such as wikipedia, dbpedia @xcite or the kbs used in the now decade-old tac-kbp challenges @xcite . as a result, previous work always assumes complete knowledge of the schema of the target kb that entity linking models are trained for, i.e. how many and which attributes are used to represent entities in the kb. this allows training supervised machine learning models that exploit the schema along with labeled data that link mentions to this a priori known kb. however, this strong assumption breaks down in scenarios which require linking to kbs that are not known at training time. for example, a company might want to automatically link mentions of its products to an internal kb of products that has a rich schema with several attributes such as product category, description, dimensions, etc. it is very unlikely that the company will have training data of this nature, i.e. mentions of products linked to its database. our focus is on linking entities to unseen kbs with arbitrary schemas. one solution is to annotate data that can be used to train specialized models for each target kb of interest, but this is not scalable. a more generic solution is to build entity linking models that work with arbitrary kbs. we follow this latter approach and build entity linking models that link to target kbs that have not been observed during training. 1 our solution builds on recent models for zero-shot entity linking @xcite . however, these models assume the same, simple kb schema during training and inference. we generalize these models to handle different kbs during training and inference, containing entities represented with an arbitrary set of attribute-value pairs. this generalization relies on two key ideas. first, we convert kb entities into strings that are consumed by the models for zero-shot linking. central to the string representation are special tokens called attribute separators, which represent frequently occurring attributes in the training kb(s), and carry over their knowledge to unseen kbs during inference (section 4.1). second, we generate more flexible string representations by shuffling entity attributes before converting them to strings, ## background conventional entity linking models are trained and evaluated on the same kb, which is typically wikipedia, or derived from wikipedia @xcite . this limited scope allows models to use other sources of information to improve linking, including alias tables, frequency statistics, and rich metadata. beyond conventional entity linking there have been several attempts to go beyond such conventional settings, e.g. by linking to kbs from diverse domains such as the biomedical sciences @xcite and music @xcite or even being completely domain and language independent @xcite . @xcite discuss approaches to link entities to a kb that simply contains a list of names without any other information. @xcite use databaseagnostic features to link against arbitrary databases. however, their approach still requires training data from the target kb. in contrast, this work aims to train entity linking models that do not rely on training data from the target kb, and can be trained on arbitrary kbs, and applied to a different set of kbs. @xcite also do unsupervised entity linking by generating rich context representations for mentions using abstract meaning representations @xcite , followed by unsupervised graph inference to compare contexts. they assume a rich target kb that can be converted to a connected graph. this works for wikipedia and adjacent resources but not for arbitrary kbs. @xcite introduce a novel zeroshot framework to "develop entity linking systems that can generalize to unseen specialized entities". contextualized representations for entity linking models in this work are based on bert @xcite . while many studies have tried to explain the effectiveness of bert for nlp tasks @xcite , the work by @xcite is most relevant as they use probing tasks to show that bert encodes knowledge of entities. this has also been shown empirically by many works that use bert and other contextualized models for entity linking and disambiguation @xcite @xcite @xcite . ## linking to unseen knowledge bases the models in section 3 were designed to operate in settings where the entities in the target kb were only represented using a textual description. for example, the entity douglas adams would be represented in such a database using a description as follows: "douglas adams was an english author, screenwriter, essayist, humorist, satirist and dramatist. he was the author of the hitchhiker's guide to the galaxy." however, linking to unseen kbs requires handling entities with an arbitrary number and type of attributes. the same entity (douglas adams) can be represented in a different kb using attributes such as "name", "place of birth", etc. (top of figure 1 ). this raises the question of whether such models, that harness the power of pre-trained language models, generalize to linking mentions to unseen kbs, including those without such textual descriptions. this section presents multiple ideas to this end. ## conclusion the primary contribution of this work is a novel framework for entity linking against unseen target kbs with unknown schemas. to this end, we introduce methods to generalize existing models for zero-shot entity linking to link to unseen kbs. these methods rely on converting arbitrary entities represented using a set of attribute-value pairs into a string representation that can be then consumed by models from prior work. there is still a significant gap between models used in this work and schema-aware models that are trained on the same kb as the inference kb. one way to close this gap is by using automatic table-to-text generation techniques to convert arbitrary entities into fluent and adequate text @xcite @xcite @xcite . another promising direction is to move beyond bert to other pre-trained representations that are better known to encode entity information @xcite @xcite . finally, while the focus of this work is only on english entity linking, challenges associated with this work naturally occur in multilingual settings as well. just as we cannot expect labeled data for every target kb of interest, we also cannot expect labeled data for different kbs in different languages. in future work, we aim to investigate how we can port the solutions introduced here to multilingual settings as well develop novel solutions for scenarios where the documents and the kb are in languages other than english @xcite @xcite .