## 3 Chinese Network Security Named Entity Recognition Method Based on CSBERT-IDCNN-BiLSTM-CRF
### Theoretical Analysis of the CSBERT-IDCNN-BiLSTM-CRF Model  
#### 3.1.1 Continue Training CSBERT in the Field of Network Security  
Deep learning requires a large amount of annotated data to train models, but it is difficult to obtain such data in professional domains. On the other hand, obtaining unannotated data is relatively easy. How to use unsupervised learning to improve the performance of small sample supervised learning is a research question worth exploring [28]. The BERT model has powerful fitting ability and can enhance the performance of small sample named entity recognition tasks. Additionally, the BERT model is trained using a large amount of unlabeled text, making it highly versatile. By training a professional domain-specific BERT model with unannotated data from that domain, the performance of small-sample named entity recognition models in that domain can be improved. Compared with fine-tuning on an open-source BERT model using professional domain data and training a professional domain-specific BERT model from scratch with full-scale data, there is little difference in their performances. This is because the professional domain mainly adds numerous proprietary terms compared to the public domain. The publicly available BERT models are trained on open-domain data and have learned general knowledge about language. With this foundation, when trained on professional corpora, pre-trained models can learn specific terms in that field. Training a BERT model essentially involves projecting characters into target semantic space through spatial transformation and representing relationships between characters based on distances between spatial coordinates. Publicly available BERT models have already achieved most character projections in semantic space and cover most content in general semantic space. Continuing training for specific domains aims at projecting general semantic space onto specialized semantic space and reducing distances between specialized vocabulary items within that field. In practical applications, this means that specialized vocabulary items are more likely to be represented as related entities rather than individual characters alone. By leveraging this pre-learned \"knowledge\" about specific domains, better performance can be achieved in small sample named entity recognition tasks within those domains. Moreover, since fine-tuning occurs on existing models without requiring full-scale data training, it reduces the resource demands of model training and lowers the threshold for individuals to apply BERT models in professional domains. In this paper, we address the development of a knowledge graph for the cybersecurity domain within a Chinese-language environment. To assess model performance, we input Chinese cybersecurity-specific terminology and general locational nouns into both the BERT and CSBERT models to generate word vectors. We performed a clustering analysis on the output vectors from these models. For visual distinction in the Chinese domain, entities related to cybersecurity are color-coded in purple, while all other entities are marked in yellow. The clustering outcomes for both models are as Figures 2 and 3. From the above two clustering result graphs, the BERT model is trained using publicly available corpora and has learned some prior knowledge. However, the word vectors outputted by the model do not show obvious clusters, indicating a poor ability to recognize network security terms. On the other hand, the CSBERT model is further trained using network security corpora, resulting in distinct distances between word vectors. Network security terms and location names are clustered separately with a significant distance between these two clusters. The CSBERT model can identify more network security terms compared to BERT. Additionally, the distances between network security vocabulary are closer in CSBERT's output word vectors, which contain more network security information than the other. Figure 3: CSBERT model clustering results in the Chinese environment.