# A Novel Named Entity Recognition Algorithm for Hot Strip Rolling Based on BERT-Imseq2 seq-CRF Model
## 3 Model Design
### Seq2 seq Model Improved  
The seq2 seq model belongs to a common Encoder-Decoder structure. The model successfully applies deep learning networks to tasks such as translation or question answering, and achieves impressive performance in conversational robots, machine translation, and speech recognition applications [21; 22].  
The main idea of the seq2 seq model is to use two neural networks as an encoder and a decoder, respectively. The encoder is responsible for converting the input sequence into a vector of a specified length. The vector can be regarded as the semantic information of the input sequence. This process is called encoding. Generally, the hidden state of the last unit is taken as the semantic vector \(C\). The decoder is responsible for generating an output sequence of a specified length based on the semantic vector \(C\). This process is called decoding. The decoder input includes two forms: in Figure 1a, the semantic vector \(C\) obtained by the encoder is added to the decoder network as the initial state, the semantic vector \(C\) only participates in the operation as the initial state, and then the implicit state of the previous unit continues to propagate downward as the implicit state input of the unit. Figure 1b shows that the semantic vector \(C\) is involved in the operation of the decoder at all times, and the implicit state of the previous unit is still the implicit state input of the current unit.  
Inspired by the seq2 seq model, NER can be regarded as an equal length sequence to sequence 'translation' task. However, the information contained in semantic vector \(C\) is too simple and general. It is far from enough to rely on semantic vector \(C\) to accurately predict sequence labels. In order to give full play to the information obtained by the encoder and the characteristics of the decoder itself, the paper improves the original seq2 seq model, recorded as Imseq2 seq, and the model structure is shown in Figure 2.  
The length of the input sequence in Figure 2 is 4, and the sequence vector is quantized and sent to the encoding layer, and the semantic vector and preliminary prediction results are obtained through the encoder. To solve the problem in which the semantic vector cannot support entity label prediction, this model adds the encoder prediction results to the decoder, and combines a Teacher-Forcing mechanism to randomly correct the labeling results of the previous unit, and then sends them to the current decoding unit. Compared with the original model, the improved seq2 seq makes full use of the output information of the encoder, and adds a Teacher-Forcing mechanism to the decoder, which helps to improve the model speed and labeling accuracy. In the actual model design, in order to ensure the consistency of the decoder data input, a 'START' vector is inserted at the beginning of each sentence. BiLSTM is used as encoder to perform preliminary prediction and vector compression on input information. The decoder uses a unidirectional LSTM to process and label the input information orderly.  
Figure 1: Typical Seq2 seq structure.