## 2 Background
### Related Work
#### 2.2.4 Word2 Vec + TextCNN  
In the proposed model, we combine Word2 Vec (a feature-rich word vector for training) with TextCNN (a neural network specifically designed for text classification), in order to further improve the classification accuracy. Word2 Vec, a set of word-embedding tools invented and pioneered by Google in 2013 [35], has a wide range of applications [36] and provides a more efficient means of representing semantic distances between words through a deep learning-based tool that obtains a vector representation of each word in a corpus by calculating the cosine distance between word vectors. Word2 Vec consists of two word vector training models, CBOW and Skip-gram, both of which include an input layer, a projection layer, and an output layer. The CBOW model predicts the current word from its context, which is suitable for use with a smaller data set; therefore, in this paper, we use the CBOW model. In contrast, the Skip-gram model predicts the current word in context. The structures of these two models are shown in Figure 7.  
Word2 Vec processes text vectors by taking into account the relationship between contexts, resulting in a more feature-rich text vector than the previous embedding method, making it easier to extract features and perform some related processing. The word vectors generated by Word2 Vec are less dimensional than those generated by embedding, and are faster and more general in operation, allowing them to be used in a variety of NLP tasks and to achieve better results in classifying data sets [37; 38]. We also use TextCNN, which is based on a modified CNN architecture. CNNs have a wide range of applications in the field of deep learning, due to their great usefulness [39; 40]. In the CNN, the data is first fed into the input layer of the CNN to obtain the original matrix. The features of this matrix are then extracted by the convolutional layer; the formula for calculating the output results is given in Formula (5). After that, the output results are obtained. As the calculation between adjacent layers of the neural network only can be conducted linearly, it is necessary to use an activation function to carry out the non-linear operation, which enables the neural network to simulate more complex models. There are two activation functions that are mainly used in traditional neural networks: \(\sigma(x)\) and \(\tanh(x)\). In the process of using these two activation functions, there are problems related to their small interval range and gradient vanishing. To solve these problems, the ReLU function is mainly used, as shown in Formula (6), which is a linear operation with high efficiency. The pooling layer is used for down-sampling (i.e., sparse processing) of feature data. The pooling formula is given in Formula (7). \[N_{2}=\frac{(N_{1}+2P-F)}{\text{stride}+1} \tag{5}\]  
where \(N_{2}\) is the size of the output, \(N_{1}\) is the size of the input data, \(F\) is the size of the convolution kernel, stride is the sliding step of the convolution kernel, and \(P\) is used to fill in the input data, allowing it to be divisible when the stride is greater than 1;  
Figure 7: CBOW and Skip-gram structure diagrams. \[f(x)=\max(0,X) \tag{6}\]  
where it can be seen that the function takes a value of 0 when the gradient is less than 0, while the actual number is taken when the gradient is greater than 0, which avoids the problem of the gradient vanishing;  
\[N_{2}=\frac{(N_{1}-F)}{\text{stride}+1} \tag{7}\]  
TextCNN is an algorithm for classifying text using a convolutional neural network, proposed by [PERSON] in 2014 [41]. TextCNN introduces some deformations into the input layer of the CNN, in order to improve the text classification performance. The data are first encoded as a vector matrix, then convolved and max-pooled. Finally, the data are classified by Softmax. TextCNN is able to automatically combine and filter N-gram features to obtain semantic information at different levels of abstraction when convolving the text, and is able to extract richer text features.