# Sentiment Analysis of Online New Energy Vehicle Reviews
## 3 The Proposed Model EDC
### ERNIE Model
By applying linear transformations according to Equations (2) and (3), we obtain a modified positional encoding \(P\):  
\[P(p_{os},2i)=\sin(p_{os}/1000^{2i/d_{model}}) \tag{2}\]  
\[P(p_{os},2i+1)=\sin(p_{os}/1000^{2i+1/d_{model}}) \tag{3}\]  
In the equations, \"\(p_{os}\)\" represents the position of a word in the text, and \"\(i\)\" denotes the corresponding vector dimension. Attention mechanism: The attention mechanism allocates different weights to different input components based on their correlation with other parts of the input data. The attention mechanism can be mathematically represented using Equations (4) and (5), which illustrate how the weights are computed and applied. This approach enables the model to dynamically adjust its focus and selectively attend to the most relevant information, making it a widely adopted and effective technique in various fields, including natural language processing and computer vision. \[\left\{ \begin{array}{l}Q=Linear(X)=X_{w_{Q^{\prime}}}\\ K=Linear(X)=X_{w_{K}},\\ V=Linear(X)=X_{w_{V}},\end{array} \right. \tag{4}\]  
In Equation (4) [36], the term \"_Linear_\" refers to a linear mapping operation that applies a linear transformation to the input text vector, denoted as \(X\), using the associated weight parameters. More specifically, the weight parameters \"_WQ_\", \"_WK_\", and \"_WV_\" are utilized to linearly transform the query, key, and value, correspondingly. By performing these linear  
Figure 4: Transformer encoder structure. transformations, we obtain weighted representations of the query, key, and value, which are subsequently utilized to compute the attention weights. \[X_{attention}=Attention(Q,K,V)=Soft\max(\frac{QK^{T}}{\sqrt{\text{d}_{k}}})V \tag{5}\]  
In the given formula, \(Q\) represents the query vector, \(K\) represents the key vector, and \(V\) represents the value vector. \(d_{k}\) represents the dimensions of the vectors. The formula involves calculating the dot product between the query vector and the key vector (\(QK^{T}\)), dividing the result by the square root of \(\sqrt{d_{k}}\), applying the softmax function to obtain attention weights, and, finally, multiplying the attention weights with the value vector. This process generates a weighted representation that effectively captures the significance of different elements based on their relevance to the query. Layer normalization and residual connections: Layer normalization and residual connections were proposed [37] to enhance the training process and performance of neural networks. Layer normalization, as depicted in Equation (6), normalizes the hidden layer to adhere to a standard normal distribution. This technique proves advantageous for handling variable-length sequence data and expediting model convergence during mini-batch training. On the other hand, residual connections, represented by Equations (9) and (10), tackle the issues of gradient vanishing and network degradation by introducing skip connections across layers. By incorporating residual connections, information can easily propagate between layers, facilitating more efficient training and improving network performance. \[LayerNorm(x)=a\odot\frac{x_{ij}-u_{i}}{\sqrt{\sigma_{i}^{2}+\varepsilon}}+\beta \tag{6}\]  
In the equations, \"\(x_{i}\)\" represents an element of the input feature vector, indicating that each element in the input feature vector is denoted by \"\(x_{i}\)\". \"\(u_{i}\)\" represents the row-wise mean of the matrix computed in Equation (7), while \"\(\sigma_{i}^{2}\)\" represents the row-wise variance of the matrix calculated in Equation (8). The element-wise multiplication in the matrix is denoted by \"\(\odot\)\", signifying that each element is multiplied individually. During model training, parameters \(\alpha\) and \(\beta\) are used for additional computations, while \"\(\epsilon\)\" is a small value added to the denominator to prevent division by zero.