Le benchmarking de la reconnaissance d’entités nommées pour le français (Benchmarking for F rench NER ) (2018)

Abstract: Cet article présente une tâche du benchmarking de la reconnaissance de l’entité nommée (REN) pour le français. Nous entrainons et évaluons plusieurs algorithmes d’étiquetage de séquence, et nous améliorons les résultats de REN avec une approche fondée sur l’utilisation de l’apprentissage semi-supervisé et du reclassement. Nous obtenons jusqu’à 77.95%, améliorant ainsi le résultat de plus de 34 points par rapport du résultat de base du modèle.

## improving ner models using semi-supervised learning we employ the ner model described in the previous section ( §2.3) to improve ner results using semi-supervised learning, in which we automatically annotate a large monolingual corpus. this kind of practice is often called self-training @xcite , self-taught learning @xcite , and lightly-supervised training @xcite . for semi-supervised learning we introduce the consensus method d @xcite . we use it by intersection between entity-annotated results using available at http://www.llf.cnrs.fr/gens/abeille/french-treebank-fr.php 12. available at http://www.statmt.org/wmt15/training-parallel-nc-v10.tgz 13. available at http://www.statmt.org/moses size (d) size ( d) per loc org (original train) 0.16 m -1 0.89 0.42 french treebank 0.62 m 0.38 m 1 0.74 0.18 news commentary 6.09 m 3.65 m 1 3.00 0.23 ## improving results using reranking we also propose a reranking algorithm using @xmath0, ..., l n ) where l i is an assigned label by a learning algorithm, and l is a reranked label by the rerank function. we exclude l hmm(t) , and we then obtain l from rerank(l crf (+) , l crf (w) , l bilstm ) for reranking labels. we calculate the rerank function as follows : there were efforts to create corpora annotated in named entities for french. @xcite and @xcite manually annotated named entities in the french treebank, and in restricted domain such as oral dialogs recored by the edf call center for information extraction, respectively. okinina et al. (2013) enriched proper nouns by mining wikipedia with the combination of dbpedia rules and a support vector machine classification. @xcite used a cross-lingual approach by converting a rule-based english ner system into french by using lexical and grammar adaptations. @xcite employed ner for better classification results on opinion mining and sentiment analysis. @xcite corrected ocred named entities errors by using a rule-based ner system. @xcite used ner for recognizing geographical references. these are applications, in which ner results improved other natural language processing tasks. otherwise, @xcite proposed a pipeline for french ner based on wapiti. ## conclusion in this paper, we trained and evaluated several sequence labeling algorithms to perform benchmarking for french named-entity recognition data. we then improved ner results using semi-supervised learning and reranking. we obtained up to 77.95%, in which we improved the result by over 34 points compared to the baseline results of the hmm. while incorporating unlabeled data into a new model is a simple method, it would not be surprising that self-training is not normally effective because errors in the original model can be amplified in the new model @xcite . we discard the weakest learner's results for the consensus method. this decision actually improves the ner results. for example, while hmm (t) obtains only 47.35% with intersection of all data for the french treebank, it achieve 50.34% by excluding d(m hmm(t) ) for data intersection. this semi-supervised process can be iterated, and it can be performed over other sets of unlabeled data for french. we assume that iterating the semi-supervised process and using a larger unlabeled data can improve ner results. we leave this to future work. however, while learning models for hmm and crfs are relatively quick, we note that training bi-lstm using a large annotated corpus (e.g. over 3.65m tokens in news commentary) takes several days even on a gpu for a single iteration. reranking basically selects the best result from the set of ner results for each sentence to have constructed high-performance nlp systems such as parsing @xcite . combining reranking and self-training is not new, which has been, for example, already proposed for syntactic parsing @xcite . while reported results show a minor improvement (e.g. we obtain 76.41% using ranking baseline, compared to 76.26% in the best baseline result), it is cheap and easy to implement for immediate improvements. comparison of results using previously proposed ner systems for french, and benchmark learning using other previously proposed ner data would be an interesting task, and we leave this to future work. all trained models and data will be publicly available at https://github.com/ jungyeul/taln2018.