# Research on Intelligent Perception Algorithm for Sensitive Information
## 4 Experiment
### Experiment Result  
We compared the KG-ERNIE model with the schemes proposed by other researchers and used the TSIIP algorithm for sensitive information detection. As shown in Table 3, we compared the KGDecteor model proposed by [PERSON] et al. [26] and the CNN-BiLSTM model proposed by [PERSON] et al. [21] and the TextCNN model proposed by [PERSON] et al. [32] and the Bi-LSTM model proposed by [PERSON] et al. [25]. The comparison results prove that the KG-ERNIE+TSIIP scheme outperforms the other methods in terms of recall, F1 score, and F2 score. Among them, the accuracy reached 0.925, the recall reached 0.951, which is 1% better than the CNN-BiLSTM model, the F1 score reached 0.938, which is 0.6% better \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Datasets** & **Training Sets** & **Validation Sets** & **Test Sets** & **Total** \\ \hline THUNews & 37,800 & 12,600 & 12,600 & 63,000 \\ JWBD & 2998 & 998 & 998 & 4994 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Experimental dataset. \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Value** \\ \hline Optimizer & Adam \\ Learning rate & \(4\times 10^{-4}\) \\ Epoch & 30 \\ Batch size & 32 \\ Active function & Relu \\ Dropout & 0.5 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Parameters of the experiment. than the KGDectector model, and the F2 score reached 0.946, which is 1% better than the CNN-BiLSTM model. The results were as we had previously expected, sacrificing some accuracy but improving recall, with the best experimental results for recall, F1 scores, and F2 scores. Then, in order to study the influence of word embedding models on the experimental results and to determine whether the word embedding model used has a greater impact on the semantic integrity extraction of the text, we replaced the word embedding ERNIE model in the text encoder with the BERT model [11], the Glove model [7], and the Word2 vec model [6] for experimental comparison. The results show that the word embedding ERNIE model used in this paper gets better experimental results in terms of recall and F1 score and F2 score, with the recall reaching 0.951, the F1 score reaching 0.938, and the F2 score reaching 0.946, as shown in Table 4. The results are also in line with our expectations, as the ERNIE model uses a masking approach that includes full word masking as well as entity masking, extracting more complete contextual semantic information and, therefore, outperforming the Bert model in most metrics. The accuracy of the ERNIE model is slightly lower than that of the Bert model, as we sacrificed a small amount of precision to achieve a higher recall, where the accuracy is 0.4% lower than the Bert model, but the recall is 2.3% higher than the Bert model. To verify the stability of the model, we use accuracy and recall as indicators and judge the stability of the model based on the changes in these indicators. The experimental results are shown in Figure 6. It can be seen that the accuracy and recall of the KG-ERNIE+TSIIP solution grow gradually as the number of training times increases, and the changes in accuracy and recall tend to stabilize after about the 15 th epoch. This shows that the solution of KG-ERNIE+TSIIP has strong stability. To demonstrate the generalization of our method, we used 2044 private data from a company as a test, including 1842 non-sensitive texts and 202 sensitive texts. The results are shown in Table 5, where the precision and recall, as well as F1 score and F2 score are lower than in Table 3. According to these results, we analyze that the sensitive words and sensitive phrases in the training dataset cannot cover all the sensitive words and sensitive sentences in the real data because the construction of the sensitive sentences library and the sensitive vocabulary are not perfect and the scale of the data is insufficient.