## 3 Method
### Embedding Layer
#### 3.1.2 Word Level Features  
The further incorporation of word-level features in the character-level feature-based model helps to improve the recognition effect of the model. In this paper, three word-level features are selected, which are word vector features, word length features, and pos features. Because word-level feature extraction requires the use of word lists, the shipping enterprise credit word list in this paper is obtained by jieba word separation and removal of duplicate words, and no additional user dictionaries are introduced in the word separation process. In order to avoid the influence of word separation errors on extracting word-level features, this paper introduces four sets B, M, E, and S to save the different words corresponding to the characters. Firstly, all the words that the characters in the sentence may form are identified according to the generated word list, and then the characters are added to the four sets of B, M, E, and S according to their positions in the words, with B representing characters appearing at the beginning of the words, M representing characters appearing in the middle of the words, E representing characters appearing at the end of the words, and S representing characters that can be seen as words themselves. When there is no matching word in the set, it is filled with the token NULL. The set of four words corresponding to each character can be expressed by the following Equations (1)-(4):  
\[B(c_{i})=\left\{w_{i,k},\ i<k\leq n\right\} \tag{1}\]  
\[M(c_{i})=\left\{w_{j,k},1\leq j<i<k\leq n\right\} \tag{2}\]  
\[E(c_{i})=\left\{w_{j,i},1\leq j<i\right\} \tag{3}\]  
\[S(c_{i})=\left\{c_{i}\right\} \tag{4}\]  
(1) Word Vectors Features  
Word features are the features of words themselves, which are the most basic features in the field of natural language processing and are represented in the form of word vectors. The word vectors in this study are the vectors after word reorganization based on characters  
Figure 2: BERT model structure. \({}^{\star}\) Notes: The English translation of the input layer is “Listed as business exception”. after stuttering and splitting. The word vectors in this study are embedded directly through the embedding layer. The four set representations corresponding to each character in the text sequence are counted, and the set vector representation corresponding to each set is calculated by weighted averaging, and the vectors of the four sets are stitched together as the word vector representation corresponding to that character, as shown in Equation (5). \[x_{i}^{v}=v_{B(\epsilon_{i})}\oplus v_{M(\epsilon_{i})}\oplus v_{E(\epsilon_{i })}\oplus v_{S(\epsilon_{i})} \tag{5}\]  
where \(x_{i}^{v}\) denotes the word vector corresponding to character \(i\) and \(v_{B(\epsilon_{i})}\) denotes the vector corresponding to the set of words \(B\). (2) Pos Features  
Chinese pos characteristics include 12 categories such as nouns, verbs, prepositions, and quantifiers. In the texts related to shipping company entities and risk penalty entities or honor entities, verbs such as \"obtain\" and \"subject to\" often appear, and the pos characteristics can effectively assist in the recognition of company entities and honor entities as well as risk penalty entities in shipping enterprise credit texts. The recognition of corporate entities and honorary entities and risk penalties in shipping enterprise credit texts can be effectively assisted according to pos features. Count the four set representations corresponding to each character in the text sequence, label all words using the pos annotation system after stuttering and splitting, calculate the pos vector representation corresponding to each set by weighted averaging, and stitch the lexical vectors of the four sets as the pos vector representation of the word corresponding to that character.