# Extracting Domain-Specific Chinese Named Entities for Aviation Safety Reports: A Case Study
## 2 Related Work
### Approaches Based on Deep Learning  
In recent years, several approaches have been proposed which attempt to utilize deep learning, achieving fruitful results in various fields. The deep learning framework is versatile and does not need to consume resources in feature engineering like machine learning does. The performance of neural networks and the effectiveness of training methods have been widely recognized [15]. Relevant researchers have conducted a lot of research on named entity recognition using neural networks. The named entity recognition framework based on deep learning is mainly divided into three parts. The first is the embedding layer, which converts text into a vector form that can be recognized by the computer. Commonly used methods include Word2 Vec [16], Glove [17], etc. The second part is the encoding layer, which obtains contextual dependencies based on the network structure and extracts hidden meanings in a text. Most of the current methods are based on RNN, CNN and transformer [18] structures. The third part is the label sequence decoding layer, which predicts and classifies characters in a text. The most widely used method is CRF. For example, [PERSON], [PERSON] [19] proposed the extraction of medical named entities from Chinese electronic medical records (CEMR) based on the ELMo-ET-CRF model. In addition, [PERSON] [20] proposed GlobalPointer based on the global pointer idea, which realizes the indiscriminate processing of nested and non-nested NER. It can achieve effective results in both non-nested and nested experiments. The most commonly used named entity encoding and decoding structure is the BiLSTM-CRF structure [21], and most of the current research on named entity recognition is also based on this model structure. For example, the paper in [22] proposed a clinical named entity recognition model combining multi-head self-attention, BiLSTM neural networks and conditional random fields.  
The pre-trained model has a relatively high status in current natural language processing (NLP). The development of pre-training models has benefited from the advancement of various technologies, including the improvement of computing hardware, the availability of big data, the development of representation learning, the improvement of model architecture, and the advancement of multi-modal data fusion algorithms. The pre-trained model is a model that is stored after training in a large corpus via the consumption of large resources. The model can be fine-tuned and used in various basic tasks of NLP [23]. In recent years, several pre-training models, such as bidirectional encoder representation from transformers (BERT) [24] and generative pre-trained transformer (GPT) [25], have been proposed which have been widely used in NLP. In the named entity recognition task, the Bert pre-training model or the Bert-based pre-training model embedding to generate word vectors has been proven to achieve better results in many studies [26; 27]. The traditional word vectors Word2 Vec and GloVe are both based on statistical methods. Although these methods can capture the semantic information of words to some extent, they often have certain limitations, such as an inability to capture complex relationships between words. In contrast, both BERT and GPT models adopt the transformer architecture and are trained via a self-attention mechanism. The transformer architecture enables the model to capture complex relationships between words and uses a large number of unsupervised learning methods for training, which improves the quality of vectors.