J eux D e L iens: Word Embeddings and Path-Based Similarity for Entity Linking using the F rench J eux D e M ots Lexical Semantic Network (2018)

Abstract: Entity linking systems typically rely on encyclopedic knowledge bases such as DBpedia or Freebase. In this paper, we use, instead, a French lexical-semantic network named JeuxDeMots to jointly type and link entities. Our approach combines word embeddings and a path-based similarity resulting in encouraging results over a set of documents from the French Le Monde newspaper.

## introduction there is an exponential growth of textual content made available on the web, produced by anyone via a broad diversity of publishing platforms. automated solutions to extract actionable insights from this content is therefore of utmost importance. focusing on textual content, we identified four main challenges that the nlp community is tackling for performing an entity linking task : dealing with different genres (social media, video subtitles, newswire articles, search queries), written in different languages, mentioning entities related to a variety of domains that can be classified with diverse sets of per-domain classes, and disambiguated against multiple kind of datasources (knowledge bases, relational databases, lexical-semantic networks) @xcite . information extraction aims to get structured information from unstructured text by attempting to interpret natural language text to extracting information about entities, relations among entities and linking entities to external referents. in detail, entity recognition aims to locate and classify entities in text into defined classes such as person, location or organization. entity linking (or entity disambiguation) aims to disambiguate entities in text to their corresponding counterpart, referred as resource, contained in a knowledge base. each resource represents a real world entity with a specific identifier. many knowledge bases can be used for doing entity linking : dbpedia @xcite , freebase @xcite , wikidata @xcite to name a few. those knowledge bases are known for being broad in terms of coverage, while vertical knowledge bases also exist in specific domains, such as geonames @xcite , musicbrainz @xcite , linkedmdb @xcite , and here jeuxdemots. we have decided to focus our work on linking entities using a lexical-semantic network as a referent base. we have identified four main lexical-semantic networks for the french language : wolf @xcite , jeuxdemots @xcite , @xcite and babelnet @xcite , the latter being multilingual. we have selected the jeuxdemots network @xcite as it offers daily updates and it is well-suited for disambiguation thanks to its polysemy representation. this network is an oriented graph where the nodes can be labeled with terms, concepts or any kind of textual item. the edges are oriented, weighted and indicate a specific type of relation between two vertices. they can be lexical (lemma, locution, action to verb, etc.) or semantic (hypernymy, meronymy, agent, etc.). so far, the jeuxdemots network has more than 1.2 million nodes, 67 million edges and about 100 relationship types. the reminder of the paper is structured as follows : section 2 detail the relevant previous work already done. section 3 introduces our two steps approach for entity linking. next, the section 4 proposes an evaluation of this approach. finally, we conclude and discuss some future work in section 5. ## related work currently, for the french language, the state of the art for entity linking suffers from a lack of evaluated approaches, resources and datasets. one of the most recent entity linking method @xcite relies on a statistical model trained on a manually annotated corpus and uses features computed using a basic heuristic tool and extracted from a large knowledge base. unfortunately, this method is hard to reproduce since the knowledge base and the corpus used in the experiments are not publicly available. we describe two methods that can be applied to french : babelfy @xcite and dbpedia spotlight @xcite . babelfy proposes a graph-based approach where two main algorithms have been developed : random walk and a heuristic for finding the subgraph that contains most of the relations between the recognized mentions and candidates. the nodes are pairs (mention,entity) and the edges correspond to existing relationships in babelnet @xcite that are scored. next, a semantic graph is built using word sense disambiguation (wsd) that extracts lexicographic concepts and does entity linking by matching strings with resources described in a knowledge base. in contrast, dbpedia spotlight relies on the so-called tf*icf (term frequency-inverse candidate frequency) score computed for each entity. the goal of this score is to show that the discriminative strength of a mention is inversely proportional to the number of candidates it is associated with. this means that a mention that commonly co-occurs with many candidates is less discriminative. although, those methods can be applied on french documents, they have never been thoroughly evaluated with this language due to a lack of proper benchmark datasets. next, those methods do not take into account the different semantics that an entity might have (see in section 3 an example with paris). ## jeuxdeliens approach we have originally developed a named entity recognition (ner) model trained with the etape @xcite corpus for recognizing entities in french documents. however, the corpus required a lot of pre-processing that was consisting of cleaning the transcripts (speech-to-text dataset). in fact, even after the cleaning the performance of the ner was very low. the extraction part not being our focus, we have assumed to take as input the surface forms of the entities that have to be linked and typed with the jeuxdemots network. our entity linking approach is designed in two steps : i) word embeddings, and ii) path-based similarity. as a running example, we will use the following three sentences : after this preprocessing, we have a list of words that represent the context of the document, and where each of them has its node mapped in the jeuxdemots network. the next step is to select the best entity candidate in the list and we use a word2vec @xcite embedding. 1. as of 1 january 2013, paris has more than 2.2 million inhabitants 2. in terms of trade, paris has clearly stated its will. 3. paris made her first appearances as a model in several charity events. 4. capital city 5. french government 6. firstname 7. familyname 8. nightstand 9. entities that do not appear in the data source being used ## conclusion and future work we have proposed jeuxdeliens, a two steps method for disambiguating entities in french documents using the jeuxdemots lexical semantic network. although the evaluation has been made on a single newspaper article, the results of jeuxdeliens are very encouraging and show that our approach can well disambiguate entities for french textual content. as future work, we aim to create a bigger evaluation dataset and to share it with the nlp community. we are also interested in finding a way to detect if no candidate entity matches, allowing us to spot missing data in the network. to do so, we plan to use the deep semantic similarity measure (dssm) method @xcite in order to directly compute similarities between the nodes and identify if the entity belongs to the network or not. finally, to solve the word2vec problem, we plan to use the fasttext library @xcite since it is robust against unknown vocabulary words because it uses letter-grams instead of tokens to compute embeddings. we can also improve the model by adding the full wikipedia dump into the frwac corpus training data.