BLINK with E lasticsearch for Efficient Entity Linking in Business Conversations (2022)

Abstract: An Entity Linking system aligns the textual mentions of entities in a text to their corresponding entries in a knowledge base. However, deploying a neural entity linking system for efficient real-time inference in production environments is a challenging task. In this work, we present a neural entity linking system that connects the product and organization type entities in business conversations to their corresponding Wikipedia and Wikidata entries. The proposed system leverages Elasticsearch to ensure inference efficiency when deployed in a resource limited cloud machine, and obtains significant improvements in terms of inference speed and memory consumption while retaining high accuracy.

## introduction companies that offer voip telephony products with built-in speech and natural language processing features aim to assist the customer support agents with information relevant to the content of their conversations with the customers. to be useful, such assistance should be provided in near realtime of the triggering utterance. in this paper, we demonstrate how we build a near real-time entity linking system at dialpad 1 to link the entities in business phone transcripts to a knowledge base to provide more semantically-informed assistance. the entity linking task is usually comprised of three steps: (i) detect the mentions in the given text, (ii) generate a list of candidate entities relevant to each mention, and finally (iii) link each mention to its most relevant entry in the knowledge base @xcite . note that entity linking systems used in production should provide the optimum performance in terms of both inference speed and memory consumption while being used within a limited computational budget. since there are millions of entities stored in a knowledge base, the 1 https://www.dialpad.com/ scaling issue is a major concern while developing a real-time entity linking system. the goal of this research is to develop a neural entity linking system to efficiently link product and organization type entities in business phone conversations to their respective entries in a knowledge base for information extraction. for that purpose, we present an extended version of the stateof-the-art neural entity linker, the blink model @xcite . though blink was originally proposed for entity linking on wikipedia, we extend it for entity linking on wikidata foot_0 since unlike wikipedia, the wikidata knowledge base contains information related to the entities in a structured way. thus, it allows effective extraction of relevant information for each entity. more importantly, for production deployment, we also introduce several new techniques that significantly reduce the memory requirements, computational resource usage, and the inference speed of blink. more concretely, our major contributions are stated below: prior work on entity linking mostly focused on linking named entities to unstructured knowledge bases like wikipedia, whereas the amount of work that used a structured knowledge base like wikidata is very limited @xcite . though other knowledge bases like dbpedia @xcite or yago @xcite have also been studied, the utilization of wikidata as the knowledge base to extract relevant information has gained lots of attention recently @xcite . detecting mentions (i.e., entities) in the given text @xcite is an important step for entity linking. in recent years, utilizing the neural network architecture for mention detection has been extensively studied @xcite . more recently, the impressive success of the transformer architecture @xcite @xcite in a wide range of natural language processing tasks has also inspired researchers to apply transformer models for the entity recognition @xcite step in entity linking @xcite , which results in obtaining superior performance over the previously used recurrent neural network-based models @xcite . for the candidate generation step in entity linking, early work mostly utilized various non-neural network approaches such as tf-idf or alias tables @xcite , whereas more recent work utilized dense embeddings learnt via pre-trained transformers to retrieve the relevant candidates @xcite . however, there is an important limitation while generating the candidates via pre-trained embeddings. for instance, the state-of-the-art neural entity linking model blink @xcite loads the pre-trained embeddings of all entities in wikipedia into memory. thus, it becomes inapplicable for deployment in production scenarios where the requirement is to ensure lower memory consumption. in this paper, we address this issue via storing the pre-trained embeddings in elasticsearch. moreover, we introduce new techniques that pre-compute the linking between wikipedia and wikidata to ensure efficient information retrieval, while also optimize the pre-trained models to meet the goal of deploying the proposed system in a limited computational resource setting. ## system overview to develop the entity linking system, we adopt blink, a neural entity linker that uses the transformer-based bert model @xcite and trains it on wikipedia. blink connects each mention in a given text with its respective wikipedia page based on the overall context. since wikipedia contains textual data in an unstructured format, it is difficult to extract information from it. thus, we connect blink with a structured knowledge base, wikidata, to extract information about product and organization type entities. note that we store our knowledge base as well as the embedding representation of each entity in elasticsearch. moreover, we replace the flair named entity recognition (ner) model @xcite originally used by blink with an ner model (we denote it as internal ner) trained on transcripts of business phone conversations using distilbert @xcite . we show our entity linking system in figure 1 . at first, the input text is processed by the ner model to detect the mention. then, we generate the representation for the input text using the pretrained blink embeddings, while we retrieve the relevant candidates with their embeddings from elasticsearch using the multi match query foot_2 feature of elasticsearch. finally, the embedding representations of the input text and the candidates are sent to the pre-trained blink bi-encoder to select the most relevant candidate. below, we first demonstrate our proposed entity linking system: blink with elasticsearch, followed by describing how we deploy our proposed system in production. ## experimental details in this section, we demonstrate the datasets used in our experiments and the implementation details. ## results and discussions we denote our entity linking model that utilizes multi match query (mmq) on elasticsearch (es) as blink + es mmq . here, we first discuss its performance on our business conversation data. then we conduct experiments on some academic datasets to demonstrate its generalized effectiveness. ## conclusion in this paper, we introduce an efficient, scalable version of the blink model and extend it for entity linking on wikidata. with extensive experiments, we show that our proposed system is usable for production environments within a limited budget setting since it significantly reduces memory requirements, computing resource usage, as well as the inference time while retaining high accuracy. we also effectively deploy our proposed entity linking system in a 10gb ram machine without using any gpu for near real-time inference. in the future, we will investigate how to make our entity linking system more efficient such that it can give inference in real-time (e.g., within one second). moreover, we will study how different bert-based @xcite @xcite sentence similarity models @xcite @xcite for candidate retrieval can impact the performance, while also exploring different techniques such as dimensionality reduction @xcite to optimize the space used in elasticsearch as well as the computing resource requirements.