## 3 Chinese Network Security Named Entity Recognition Method Based on CSBERT-IDCNN-BiLSTM-CRF
### Theoretical Analysis of the CSBERT-IDCNN-BiLSTM-CRF Model
#### 3.1.2 Introduction of BiLSTM-CRF Module Based on CSBERT Model  
Due to the temporal nature of natural language, it is crucial to utilize neural network models that can extract sequential features when modeling temporal information [29]. The chain-like structure of Recurrent Neural Network (RNN) naturally fits the sequence modeling problem in natural language processing. Therefore, RNN has achieved good performance in handling natural language tasks. However, with the increase in computing power and input text length, the original RNN model's ability to handle long texts becomes limited. On the other hand, RNN tends to suffer from gradient vanishing problems, leading to poor performance in extracting long-distance dependency features.  
To address the issue of long-distance dependencies, Long Short-Term Memory (LSTM) was introduced. LSTM is a special type of RNN that partially solves the problem by incorporating gate mechanisms on top of the basic RNN structure. This greatly improves the upper limit on input text length. However, since LSTM models only capture information from one direction while extracting sequential features, they cannot fully capture contextual information present in each character of natural language text. In different contexts or linguistic environments, a word may have different meanings. To better extract contextual information, this paper adds a backward LSTM on top of forward LSTM and using Bidirectional LSTM (BiLSTM) model instead. The structure of BiLSTM model is as Figure 4:  
When using a single BiLSTM, the model usually takes the label with the highest probability at each time step as its output result. The output result only depends on the output of that time step and ignores the dependency relationship between the entire label sequence. This labeling method does not consider label regularization issues, which may lead to many illegal label sequences and reduce the performance of the model. To solve this problem, a CRF module is introduced under the premise of BiLSTM. CRF considers dependencies between labels and adds some restrictions to ensure label legality in the model's output. The objective function of CRF consists of two parts and is also constrained by surrounding annotation results. It abstracts sequence labeling problems into dynamic programming problems, seeking for an optimal sequence that maximizes the objective function and reduces the probability of incorrect labels appearing.  
Figure 4: BiLSTM Structure Diagram.  
The working process of the CRF module is as shown in the formula.  
Given a sequence \(X=\{c_{1},\,c_{2},......,\,c_{n}\}\) composed of n characters and a label sequence \(Y=\{y_{1},\,y_{2},......,\,y_{n}\}\), calculate the score as shown in Formula (1).  
\[score(X,Y)=\sum_{i=0}^{n}A_{y_{i},y_{i+1}}+\sum_{i=0}^{n}P_{i,y_{i}} \tag{1}\]  
where \(P\) is the probability matrix output by the upper-level feature extraction model. \(P_{i,y_{i}}\) represents the probability of the \(i\)-th character having a label \(y_{i}\). \(A\) is a probability transition matrix generated by a CRF model during training. \(A_{y_{i},y_{i+1}}\) represents the probability of transitioning from label \(y_{i}\) to label \(y_{i+1}\). Perform SoftMax operation on all label sequences to obtain normalized probabilities, as shown in Formula (2).  
\[P(Y|X)=\frac{exp(score(X,Y))}{\sum_{Y^{\prime}}exp(score(X,Y^{\prime}))} \tag{2}\]  
Maximize its logarithmic likelihood function, as shown in the Formula (3).  
\[logP\big{(}Y^{X}|X)=score\big{(}X,Y^{X}\big{)}-log\big{(}\sum exp\big{(}score \big{(}X,Y^{\prime}\big{)}\big{)}\big{)} \tag{3}\]  
Decode using the Viterbi algorithm and select the label sequence with the highest probability as the output of CRF, as shown in Formula (4).  
\[Y^{*}=arg\underset{Y^{\prime}}{max}score\big{(}X,Y^{\prime}\big{)} \tag{4}\]  
Using the BiLSTM-CRF module can better handle word sequence labeling tasks output by the CSBERT model and complete NER tasks.