This paper demonstrates end-to-end neural network architectures for Vietnamese named entity recognition. Our best model is a combination of bidirectional Long Short-Term Memory (Bi-LSTM), Convolutional Neural Network (CNN), Conditional Random Field (CRF), using pre-trained word embeddings as input, which achieves an F1 score of 88.59 on a standard test set. Our system is able to achieve a comparable performance to the first-rank system of the VLSP campaign without using any syntactic or hand-crafted features. We also give an extensive empirical study on using common deep learning models for Vietnamese NER, at both word and character level.
Within the large body of research on NER which have been published in the last two decades, we identify two main approaches. The first approach is characterized by the use of well-established sequence labeling models such as conditional random field (CRF), hidden markov model, support vector machine, maximum entropy and so on. The performance of these models is heavily dependent on hand-crafted features. In particular, most of the participants at CoNLL-2003 shared task attempted to use information other than the available training data such as gazetteers and unannotated data. The best system at CoNLL-2003 shared task is the work of @cite_5 which achieved an @math score of 88.76 them by using phrase features extracted from an external database. Moreover, training NER models jointly with related tasks helps improve their performance. For instance, @cite_22 trained a CRF model for joint-learning three tasks, including coreference resolution, entity linking, and NER, and achieved the state-of-the-art result on OntoNotes dataset. With a similar approach, @cite_14 gained the best performance on CoNLL-2003 shared task dataset.
@cite_5 @cite_14 @cite_22
2056451646 2251559320 2293004735
Extracting named entities in text and linking extracted names to a given knowledge base are fundamental tasks in applications for text understanding. Existing systems typically run a named entity recognition (NER) model to extract entity names first, then run an entity linking model to link extracted names to a knowledge base. NER and linking models are usually trained separately, and the mutual dependency between the two tasks is ignored. We propose JERL, Joint Entity Recognition and Linking, to jointly model NER and linking tasks and capture the mutual dependency between them. It allows the information from each task to improve the performance of the other. To the best of our knowledge, JERL is the first model to jointly optimize NER and linking tasks together completely. In experiments on the CoNLL’03 AIDA data set, JERL outperforms state-of-art NER and linking systems, and we find improvements of 0.4 absolute F1 for NER on CoNLL’03, and 0.36 absolute precision@1 for linking on AIDA. State-of-the-art sequence labeling systems traditionally require large amounts of task-specific knowledge in the form of hand-crafted features and data pre-processing. In this paper, we introduce a novel neutral network architecture that benefits from both word- and character-level representations automatically, by using combination of bidirectional LSTM, CNN and CRF. Our system is truly end-to-end, requiring no feature engineering or data pre-processing, thus making it applicable to a wide range of sequence labeling tasks. We evaluate our system on two data sets for two sequence labeling tasks --- Penn Treebank WSJ corpus for part-of-speech (POS) tagging and CoNLL 2003 corpus for named entity recognition (NER). We obtain state-of-the-art performance on both the two data --- 97.55 accuracy for POS tagging and 91.21 F1 for NER. We explore the use of Wikipedia as external knowledge to improve named entity recognition (NER). Our method retrieves the corresponding Wikipedia entry for each candidate word sequence and extracts a category label from the first sentence of the entry, which can be thought of as a definition part. These category labels are used as features in a CRF-based NE tagger. We demonstrate using the CoNLL 2003 dataset that the Wikipedia category labels extracted by such a simple method actually improve the accuracy of NER. We analyze some of the fundamental design challenges and misconceptions that underlie the development of an efficient and robust NER system. In particular, we address issues such as the representation of text chunks, the inference approach needed to combine local NER decisions, the sources of prior knowledge and how to use them within an NER system. In the process of comparing several solutions to these challenges we reach some surprising conclusions, as well as develop an NER system that achieves 90.8 F1 score on the CoNLL-2003 NER shared task, the best reported result for this dataset.