# Enhancement of Question Answering System Accuracy via Transfer Learning and BERT
## 4 Method
### Model For Entity Linking
#### 4.2.1 Named Entity Recognition  
The goal of NER is to recognize named entities in text and assign them to the corresponding entity types. It is important since the semantic expression of Chinese corpus is sparse, many similar entities are difficult to be distinguished, and differences exist in the presentation of Chinese interrogative sentences and fundamental knowledge. Hence, it is difficult for general models to learn text features thoroughly, which makes it challenging to enhance the accuracy of entity linking. Here we suggest an adversarial transfer learning  
Figure 6: The overall process of the BAT-KBQA framework. (The modules of the three cores have been marked with bright colors.) NER model. Our model incorporates adversarial transfer learning and the CWS task [45] to solve these problems and introduces two major innovations, that is, applying CWS tasks brings shared information without introducing new noise and adding a self-attentive mechanism in the middle of the BiLSTM and CRF layers. The model tentatively learns word boundary information shared by the task from the CWS task, then filters particular information of CWS and unambiguously captures long range dependencies between two arbitrary characters in one sentence finally. Figure 7 presents the architecture of this model. The model is composed of five elements: embedding layer, shared-private feature extractor, self-attention, task-specific CRF, and task discriminator. Each portion of the model would be given a detailed elaboration in subsequent sections. First, the NER model is adopted to distinguish the topic entity in the question as the starting point for semantic parsing of the question, e.g., \"Journey to the West2\" in \"Who is the author of Journey to the West2\" is the reference form of the subject entity of the question. NER can be regarded as a sequence annotation task, in which the common tagging system \"BIO\" is used. Here \"B\" indicates the starting position of the entity mentioned, \"I\" indicates the middle or end positions of entity mentions, and \"O\" denotes that the character is not an entity mention. The position corresponding to the \"Journey to the West\" in the question is labelled as \"B I I I\", and the other non-entity mentions are labelled as \"O\". In this paper, the NER and CWS task datasets are labelled separately and embedded into the model for training. The embedding layer and other neural network models have similarities. Pre-trained embedding dictionaries are loaded to map discrete NER and CWS characters into distributed embedding vectors. Long-term memory [46] is a RNN's variant and it can resolve the gradient disappearance and explosion problems by introducing gate mechanisms and memory units. The unidirectional LSTM utilizes only the past information but overlooks future information, so we use BiLSTM for feature extraction to fuse the information from two sides of the sequence. In our model, in addition to the two-end BiLSTM for extracting private features,  
Figure 7: Named Entity Recognition model. (Consists of BiLSTM, Self-attention, CRF, feature recognizer). we add a shared BiLSTM to extract the boundary information shared by the NER and CWS tasks. That is, for the dataset of task \(m\), the hidden states of the shared and private BiLSTM layers can be calculated as shown in the following. \[\mathbf{s}_{i}^{m}=\text{BiLSTM}\big{(}\mathbf{x}_{i}^{m},\mathbf{s}_{i-1}^{m}; \theta_{s}\big{)} \tag{8}\]  
\[\mathbf{h}_{i}^{m}=\text{BiLSTM}\big{(}\mathbf{x}_{i}^{m},\mathbf{h}_{i-1}^{m };\theta_{m}\big{)} \tag{9}\]  
where \(\theta_{s}\) is the shared BiLSTM parameter and \(\theta_{m}\) is the private BiLSTM parameters. In the third step, we draw self-attention as applied to machine translation and semantic role labelling. After the feature extractor, we add a multi-headed attention mechanism [40] in order to learn the dependencies between two arbitrary words in one sentence and seize the internal structural information of the sentence.