# BiLSTM-CRF-KG: A Construction Method of Software Requirements Specification Graph
## 3 Structural Information Extraction Model of Functional Requirements
### Generation of Functional-Data Graph
In order to further verify the effectiveness of the BiLSTM-CRF-KG model, the experimental results of BiLSTM-CRF-KG and the other three named entity recognition models are compared and analyzed. The evaluation results of named entity recognition are shown in Table 4.

The simulation results show that:

1. From the perspective of named entity recognition alone, the performance of the BiLSTM-CRF model for the functional entity recognition of Chinese software requirements is better than the IDCNN-CRF and CRF++ models. 2. The performance of BiLSTM-CRF-KG is much better than that of BiLSTM-CRF and the other models. It is proven that the word sense disambiguation network, entity disambiguation and hidden relationship learning network greatly improve the performance of the model, and especially make up for the low recall rate of BiLSTM-CRF. #### 4.3.2 Applicability Analysis

In order to further verify the applicability of the model, experiments are carried out on data sets with different characteristics. The overall performance comparison is shown in Table 5. It can be seen that there is little difference in the performance of the BiLSTM-CRF-KG model under different data sets. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Accuracy** & **Precision** & **Recall** & **F1-score** \\ \hline IDCNN-CRF & 89.13\% & 88.90\% & 89.60\% & 89.25\% \\ \hline BiLSTM-CRF & 90.88\% & 91.20\% & 90.68\% & 90.94\% \\ \hline CRF++ & 85.94\% & 85.89\% & 87.28\% & 86.58\% \\ \hline BiLSTM-CRF-KG & **96.48\%\(\clubsuit\)** & **96.31\%\(\clubsuit\)** & **96.17\%\(\clubsuit\)** & **96.24\%\(\clubsuit\)** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison of named entity recognition models. Figure 12: Recall scatter diagram. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Accuracy** & **Precision** & **Recall** & **F1-Score** \\ \hline Data set 1 & 97.12\% & 97.40\% & 97.04\% & 97.22\% \\ \hline Data set 2 & 95.41\% & 95.59\% & 95.68\% & 95.63\% \\ \hline Data set 3 & 96.40\% & 96.27\% & 96.00\% & 96.13\% \\ \hline Data set 4 & 94.97\% & 95.43\% & 93.19\% & 94.30\% \\ \hline Overall & 96.48\% & 96.31\% & 96.17\% & 96.24 Figure 13 shows that in the named entity recognition stage, due to the large number of abbreviations and pronouns, the recall rate of the second set of data sets is far lower than the average level; the performance of the first data set is the best, much higher than the average, while the performance of the fourth data set is the worst. In order to further verify the performance of each group of data sets in different stages, simulation experiments are carried out for each data set in different stages. The specific F1-score is shown in Figure 14.

The experimental results show that the performance of each data set is improving with the deepening of the experiment. Moreover, each step of the model will significantly improve the performance of a set of data sets. For example, the model of acronym and pronoun alignment greatly improves the performance of the second set of data sets, while there are more acronyms and pronouns in the second set of data sets, which prove the necessity of knowledge alignment. The changes of each evaluation index of this data set before and after the model of acronym and pronoun alignment are shown in Figure 15.

Figure 14: F1-score of each data set in different stages. Figure 13: Model performance of named entity recognition in different data sets. It can be seen from the performance comparison results that although the alignment of abbreviations and pronouns cannot greatly improve the accuracy of the model, the improvement of the recall rate is huge. Through the observation and analysis of the original corpus, it is found that for acronyms and pronouns, especially pronouns, the named entity recognition model is prone to omission in the recognition process, resulting in a low recall rate. At the same time, it can be found that for the hidden relationship learning steps, the performance of the fourth set of data set is better than that of the third set of data set.