HOSMEL : A Hot-Swappable Modularized Entity Linking Toolkit for C hinese (2022)

Abstract: We investigate the usage of entity linking (EL)in downstream tasks and present the first modularized EL toolkit for easy task adaptation. Different from the existing EL methods that dealwith all the features simultaneously, we modularize the whole model into separate parts witheach feature. This decoupled design enablesflexibly adding new features without retraining the whole model as well as flow visualization with better interpretability of the ELresult. We release the corresponding toolkit,HOSMEL, for Chinese, with three flexible usage modes, a live demo, and a demonstrationvideo. Experiments on two benchmarks forthe question answering task demonstrate thatHOSMEL achieves much less time and spaceconsumption as well as significantly better accuracy performance compared with existingSOTA EL methods. We hope the release ofHOSMEL will call for more attention to studyEL for downstream tasks in non-English languages.

## introduction entity linking (el) is to extract the candidate mentions in the sentences and link them to their corresponding entities in the knowledge bases (kb) such as freebase, wikidata, and dbpedia. the linked entities encode rich knowledge from the kb, which can enhance many downstream tasks such as information retrieval @xcite , recommendation @xcite , question answering @xcite , and language model pre-training @xcite . as el is usually deployed in the pre-processing stages of these tasks, an urgent demand for el models is to guarantee a high accuracy to prevent potential error propagation. existing researches have fully explored the el problem. from the matching-based methods @xcite @xcite @xcite to the generation-based methods @xcite , the sota models such as blink @xcite , genre @xcite , and entqa @xcite , have considered different features such as mention, entity subtitle, and entity description, resulting in outstanding performances on various published el benchmarks. however, the existing advanced models usually aggregate all the features for training. despite their excellent performance, they are difficult to be adapted to specific downstream tasks. figure 2 illustrates an el example for answering the question "what religion does luke's master believe in?", by which multiple mentions are detected and linked to the entities in xlore foot_0 @xcite ) -one of the largest chinese kbs. depending only on the subtitle description of an entity, the mention, "luke", can be highly probably linked to both the entity "luke skywalker" and "luke cage" foot_1 . whereas in this scenario, we can accurately find that the former better matches because it has a relation "master" in xlore which is exactly the questioned aspect. this case presents a common phenomenon that downstream tasks usually require additional information invocation such as relation for better el results. generally, developers need to annotate specific data and perform after treatments for el model adaption. (for the above example, we need to annotate a new dataset such as the one including relation as the additional feature for question answering and retrain the el model on the new dataset). however, in the era of advanced large el models, such data annotation and model retraining is quite costly and inefficient, which raises a natural question: can we develop an effective el tool that can be easily adapted to downstream tasks? presented work. we propose a hot-swappable modularized entity linking toolkit (hosmel) to solve the above problem. compared with existing el methods or toolkits, hosmel is more suitable for the downstream tasks because of its following characteristics: for flexible usage, we release three usage methods. the first one is a ready-to-use release for directly invoking the api or accessing the web application. the second one is a partial release for users who prefer to include parts of the release as a pre-step to improve the recall of their model. the third one is an easy-to-change release that enables adding additional features or training with self-defined data. we select question answering as the downstream task to evaluate the proposed hosmel. we conduct extensive experiments on two question answering benchmarks. the results reveal three major advantages: (1) the training time of the lightweight hosmel is reduced by 4-5 times compared with two sota el models, genre @xcite and @xcite , in advance, the storage occupancy rate is also reduced by 78% compared with entqa. (2) hosmel can achieve much better performance (+8.49-17.06% accuracy) than the best baseline entqa on less training data. contributions. (1) we investigate the usage of el in downstream tasks and raise the problem of adaptation for el in downstream tasks. (2) we design a hot-swappable modularized el system and release the corresponding toolkit in chinese 1 and a live demo 2 . ## problem definition problem 1. entity linking (el): given an input text @xmath0, • • • , w n } and a kb e, the output of el is a list of mention-entity pairs {(m i , e i )} k @xmath1, where each mention m i is a text span extracted from d, and each entity e i is included in e. we assume each mention has a valid gold entity in the kb and leave the out-of-kb prediction (i.e., nil prediction) to future works. ## the proposed hosmel hosmel modularizes mention filtering, mention detection, and entity disambiguation by each entity attribute separately. generally, given an input text, hosmel first selects all the possible mentions and then detects the useful ones, whose candidate entities are then measured by each attribute of them independently. apart from mention filtering, each subsequent step aggregates the scores of all the previous steps and outputs a new top-k result to its next. figure 1 illustrates the overall framework of the proposed hosmel, each step explained below. ## the usage of hosmel we consider three different usage scenarios of the proposed hosmel and release the corresponding toolkit usage scripts with a live demo. ## experiment in this section, we use two question answering benchmarks to evaluate the el capacity of the proposed hosmel and also show its ability to easily add task-relevant features that can benefit the el performance. ## related work el has attracted lots of attention, and many methods have been studied. among them, matchingbased methods @xcite @xcite @xcite and generation-based methods @xcite are two mainstreams. the former ones usually use a dense retriever based on the maximum inner-product search (mips) to retrieve entity candidates, followed by a cross-encoder to re-rank them. the later ones frame el as a seq2seq model to autoregressively generate the text annotated with the entities' identifiers, such as their subtitles. however, both put all the features together for training, increasing the difficulty of adjusting for specific downstream tasks. since the downstream tasks usually require specific filtering or additional information invocation for better el results, these el models need the newly annotated dataset for retraining, which is costly and inefficient. in addition, most of them are designed for english, and only a few @xcite have been released as a ready-to-use toolkit. hosmel is an el toolkit for chinese that can easily adjust to downstream tasks. ## conclusion and future work we release a chinese el toolkit hosmel, which has shown to be an effective, efficient, and inter-pretable method due to the hot-swappable modulized structure. moreover, for adapting to the downstream tasks, hosmel can be easily improved by training on additional features with limited training data. experiments on two question answering benchmarks have demonstrated the time/space efficiency and the effectiveness compared with the sota el models, as well as the easy task adaptation ability. an english version of the toolkit is planned to be released in the future. yang, kyle richardson, and zhenzhong lan. 2020. clue: a chinese language understanding evaluation benchmark. in proceedings of the 28th international conference on computational linguistics, pages 4762-4772, barcelona, spain (online). international committee on computational linguistics. ikuya yamada, koki washio, hiroyuki shindo, and yuji matsumoto. 2019. global entity disambiguation with pretrained contextualized embeddings of words and entities. arxiv preprint arxiv:1909.00426. jing zhang, bo chen, lingxi zhang, xirui ke, and haipeng ding. 2021a. neural, symbolic and neuralsymbolic reasoning on knowledge graphs. ai open, 2:14-35. shiyue zhang, benjamin frey, and mohit bansal. 2021b. chrentranslate: cherokee-english machine translation demo with quality estimation and corrective feedback. in proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing: system demonstrations, pages 272-279. wenzheng zhang, wenyue hua, and karl stratos. 2021c. entqa: entity linking as question answering. zhengyan zhang, xu han, zhiyuan liu, xin jiang, maosong sun, and qun liu. 2019. ernie: enhanced language representation with informative entities. in proceedings of the 57th annual meeting of the association for computational linguistics, pages 1441-1451, florence, italy. association for computational linguistics.