# Multi-Meta Information Embedding Enhanced BERT for Chinese Mechanics Entity Recognition
## 2 Related Works  
Recent research has demonstrated that external information and a pre-trained language model can improve the performance of a CNER algorithm. These algorithms can be divided into three categories: external information-based algorithms, pre-trained language model-based algorithms, and hybrid algorithms. External information-based algorithms aim to enhance character representation by adding external information (such as lexical information). [PERSON] et al. [18] were the first to develop this type of method. They proposed Lattice LSTM, which encodes and matches words of the input sentence. However, each character of Lattice LSTM can only obtain the lexical information ending with that character, and there is no continuous memory for the previous lexical information. Moreover, [PERSON] et al. [19] proposed the word-character graph convolution network (WCGCN). This algorithm enhances the lexical information using a graph convolution network (GCN). Additionally, [PERSON] et al. [20] proposed a multi-information-based algorithm. They used a bidirectional gated recurrent unit (Bi-GRU)-conditional random field (CRF) to fuse character embedding, lexical embedding, and radical embedding. Further research was reported in [21; 22; 23; 24; 25]. However, these methods do not leverage a pre-trained language model, which could significantly improve the recognition performance. Pre-trained language model-based algorithms aim to enhance character representation using a pre-trained model (such as BERT). Research [26; 27] has proved that using the character features from BERT outperforms static embedding-based CNER algorithms. These methods usually use fine-tuning to connect the pre-trained language model with traditional CNER models, such as BERT-BiLSTM-CRF and BERT-BiGRU-CRF. A defect of these methods is that they do not consider the effect of external information on the recognition performance, which is more obvious in the vertical field. Hybrid algorithms aim to integrate external information and a pre-trained language model [17; 28; 29; 30]. [PERSON] et al. [30] proposed a model-level fusion method, named SoftLexicon. This algorithm takes the output of BERT as an enhanced character representation, directly concatenates it with the lexical embedding, and then inputs them into a fusion layer for CNER. Additionally, [PERSON] et al. [28] proposed the flat-lattice transformer (FLAT) to integrate lexical information and character embedding. Similarly, FLAT also uses model-level fusion when leveraging BERT. Although these methods simultaneously utilize external information and a pre-trained language model, they only use the model-level integration of the two, which does not allow the pre-trained language model to fully learn the external information and easily leads to overfitting. With the development of deep learning, the CNER algorithm has become more and more mature in the open domain. However, for the vertical domain, the data sources are diverse, and the data structure is different. Compared with the open domain, the entities of the vertical domain are complex and difficult to identify. [PERSON] et al. [31] proposed a CNER method combining a convolutional neural network (CNN) [32], BiLSTM, and an attention mechanism (AM) to recognize agricultural pest- and disease-related named entities. [PERSON] et al. [33] proposed a hybrid neural network model based on MC-BERT, namely MC-BERT + BiLSTM + CNN + MHA + CRF, to recognize entities in Chinese electronic medical records. [PERSON] et al. [34] proposed a CNER method that combines knowledge graph embedding with a self-attention mechanism to recognize entities in Chinese marine texts. [PERSON] et al. [26] applied BERT-BiLSTM-CRF to the recognition of named entities in the field of history and culture. However, these research works simply applied the open-domain algorithms to the vertical domain, and few works have taken into account the characteristics of professional entities.