The state-of-the-art named entity recognition ( NER ) systems are statistical machine learning models that have strong generalization capability ( i.e. , can recognize unseen entities that do not appear in training data ) based on lexical and contextual information . However , such a model could still make mistakes if its features favor a wrong entity type . In this paper , we utilize Wikipedia as an open knowledge base to improve multilingual NER systems . Central to our approach is the construction of high-accuracy , highcoverage multilingual Wikipedia entity type mappings . These mappings are built from weakly annotated data and can be extended to new languages with no human annotation or language-dependent knowledge involved . Based on these mappings , we develop several approaches to improve an NER system . We evaluate the performance of the approaches via experiments on NER systems trained for 6 languages . Experimental results show that the proposed approaches are effective in improving the accuracy of such systems on unseen entities , especially when a system is applied to a new domain or it is trained with little training data ( up to 18.3 F 1 score improvement ) . Named entity recognition ( NER ) is an important NLP task that automatically detects entities in text and classifies them into pre-defined entity types such as persons , organizations , geopolitical entities , locations , events , etc . NER is a fundamental component of many information extraction and knowledge discovery applications , including relation extraction , entity linking , question answering and data mining . The state-of-the-art NER systems are usually statistical machine learning models that are trained with human-annotated data . Popular models include maximum entropy Markov models ( MEMM ) ( McCallum et al . , 2000 ) , conditional random fields ( CRF ) ( Lafferty et al . , 2001 ) and neural networks ( Collobert et al . , 2011 ; Lample et al . , 2016 ) . Such models have strong generalization capability to recognize unseen entities1 based on lexical and contextual information ( features ) . However , a model could still make mistakes if its features favor a wrong entity type , which happens more frequently for unseen entities as we have observed in our experiments . Wikipedia is an open-access , free-content Internet encyclopedia , which has become the de facto on-line source for general reference . A Wikipedia page about an entity normally includes both structured information and unstructured text information , and such information can be used to help determine the entity type of the referred entity . So far there are two classes of approaches that exploit Wikipedia to improve NER . The first class of approaches use Wikipedia to generate features for NER systems , e.g. , ( Kazama and Torisawa , 2007 ; Ratinov and Roth , 2009 ; Radford et al . , 2015 ) . Kazama and Torisawa ( 2007 ) try to find the Wikipedia entity for each candidate word sequence and then extract a category label from the first sentence of the Wikipedia entity page . A part-of-speech ( POS ) tagger is used to extract the category label features in the training and decoding phase . Ratinov and Roth ( 2009 ) aggregate several Wikipedia categories into higher-level concept and build a gazetteer on top of it . The two approaches were shown to be able to improve an English NER system . Both approaches , however , are language-dependent because ( Kazama and Torisawa , 2007 ) requires a POS tagger and ( Ratinov and Roth , 2009 ) requires manual category aggregation by inspection of the annotation guidelines and the training set . Radford et al . ( 2015 ) assume that document-specific knowledge base ( e.g. , Wikipedia ) tags for each document are provided , and they use those tags to build gazetteer type features for improving an English NER system . The second class of approaches use Wikipedia to generate weakly annotated data for training multilingual NER systems , e.g. , ( Richman and Schone , 2008 ; Nothman et al . , 2013 ) . The motivation is that annotating multilingual NER data by human is both expensive and time-consuming . Richman and Schone ( 2008 ) utilize the category information of Wikipedia to determine the entity type of an entity based on manually constructed rules ( e.g. , category phrase " Living People " is mapped to entity type PERSON ) . Such a rule-based entity type mapping is limited both in accuracy and coverage , e.g. , ( Toral and Muoz , 2006 ) . Nothman et al . ( 2013 ) train a Wikipedia entity type classifier using human-annotated Wikipedia pages . Such a supervised-learning based approach has better accuracy and coverage , e.g. , ( Dakka and Cucerzan , 2008 ) . A number of heuristic rules are developed in both works to label the Wikipedia text to create weakly annotated NER training data . The NER systems trained with the weakly annotated data may achieve similar accuracy compared with systems trained with little human-annotated data ( e.g. , up to 40 K tokens as in ( Richman and Schone , 2008 ) ) , but they are still significantly worse than well-trained systems ( e.g. , a drop of 23.9 F 1 score on the CoNLL data and a drop of 19.6 F 1 score on the BBN data as in ( Nothman et al . , 2013 ) ) . In this paper , we propose a new class of approaches that utilize Wikipedia to improve multilingual NER systems . Central to our approaches is the construction of high-accuracy , high-coverage multilingual Wikipedia entity type mappings . We use weakly annotated data to train an English Wikipedia entity type classifier , as opposed to using humanannotated data as in ( Dakka and Cucerzan , 2008 ; Nothman et al . , 2013 ) . The accuracy of the classifier is further improved via self-training . We apply the classifier on all the English Wikipedia pages and construct an English Wikipedia entity type mapping that includes entities with high classification confidence scores . To build multilingual Wikipedia entity type mappings , we generate weakly annotated classifier training data for another language via projection using the inter-language links of Wikipedia . This approach requires no human annotation or language-dependent knowledge , and thus can be easily applied to new languages . Our goal is to utilize the Wikipedia entity type mappings to improve NER systems . A natural approach is to use a mapping to create dictionary type features for training an NER system . In addition , we develop several other approaches . The first approach applies an entity type mapping as a decoding constraint for an NER system . The second approach uses a mapping to post-process the output of an NER system . We also design a robust joint approach that combines the decoding constraint approach and the post-processing approach in a smart way . We evaluate the performance of the Wikipediabased approaches on NER systems trained for 6 languages . We find that when a system is well trained ( e.g. , with 200 K to 300 K tokens of human-annotated data ) , the dictionary feature approach achieves the best improvement over the baseline system ; while when a system is trained with little human-annotated training data ( e.g. , 20 K to 30 K tokens ) , a more aggressive decoding constraint approach achieves the best improvement . In both scenarios , the Wikipediabased approaches are effective in improving the accuracy on unseen entities , especially when a system is applied to a new domain ( 3.6 F 1 score improvement on political party articles / English NER ) or it is trained with little training data ( 18.3 F 1 score improvement on Japanese NER ) . We organize the paper as follows . We describe how to build English Wikipedia entity type mapping in Section 2 and extend it to multilingual mappings in Section 3 . We present several Wikipedia-based approaches for improving NER systems in Section 4 and evaluate their performance in Section 5 . We conclude the paper in Section 6 . In this paper , we proposed and evaluated several approaches that utilize high-accuracy , high-coverage Wikipedia entity type mappings to improve multilingual NER systems . These mappings are built from weakly annotated data , and can be easily extended to new languages with no human annotation or language-dependent knowledge involved . Experimental results show that the Wikipediabased approaches are effective in improving the generalization capability of NER systems . When a system is well trained , the dictionary feature approach achieves the best improvement over the baseline system ; while when a system is trained with little human-annotated training data , a more aggressive decoding constraint approach achieves the best improvement . The improvements are larger on unseen entities , and the approaches are especially useful when a system is applied to a new domain or it is trained with little training data .

Challenge: Even with strong generalization capability, existing state-of-the-art named entity recognition still favors a wrong entity type and makes mistakes on unseen entities.
Approach: They propose to use entity type mappings obtained by a weakly supervised classifier based on Wikipedia to improve multilingual named entity recognition systems.
Outcome: Evaluation in 6 languages shows the proposed method improves over baselines especially on unseen entities and in new domains or is trained with little data.