# Sentiment Analysis of Online New Energy Vehicle Reviews
## 3 The Proposed Model EDC
### ERNIE Model  
Early text sentiment analysis methods encounter several issues when encoding information from sentiment comments about new energy vehicles:  
1. Chinese words and phrases are semantically rich, often exhibiting polysemy and other challenges, which leads to inaccurate semantic representation in the encoded vectors. 2. Some automotive comments employ highly specialized vocabulary, making the text obscure and difficult to comprehend. The prevailing pretrained language model, BERT, has addressed problems related to the insufficient extraction of contextual semantics in traditional models. However, it still faces certain challenges in Chinese sentiment analyses of new energy vehicle comments, including unclear recognition of Chinese semantic concepts and limited Chinese representation capabilities. ERNIE [32], an improved version of the BERT model, tackles these issues by introducing a significant amount of Chinese language corpus during pretraining and optimizing the masking mechanism. It incorporates two novel masking strategies: phrase-based masking and entity-based masking, which enhance its ability to capture long-term semantic dependencies and knowledge relationships. For instance, when given the input sentence \"The design is good, I like it\", ERNIE treats the words \"design\" and \"like\" as individual units and applies a unified mask to them. This approach enables ERNIE to potentially learn more knowledge and longer semantic dependencies, resulting in better generalization of the model. This article adopts the ERNIE (Enhanced Representation through kNowledge IntEgration) model as the corpus-encoding model. The overall structure of the ERNIE model is depicted in Figure 3. In the diagram, \"Trm\" represents the Transformer encoder, which leverages the self-attention mechanism [33; 34] to quantitatively capture the inter-dependence among characters, effectively addressing contextual ambiguities. The vector representation [E1, E2,..., En] denotes the outcomes of embedding operations applied to the text within the ERNIE model. The embedding process comprises three essential components: (1) token embedding, which assigns vectors to individual characters in the token sequence and differentiates between sentences; (2) segment embedding, which distinguishes sentence boundaries in the character vector sequence and represents the sequence at the sentence level; and (3) position embedding, which signifies the position of each character in the vector sequence. Initially, the Transformer converts each word in the sentence into an embedded representation, and subsequently, the ERNIE model utilizes attention mechanisms to capture the relationships among words. As depicted in Figure 4, the input representation \(\rm{X}=(x_{1},x_{2},\ldots,x_{n})\) passes through the self-attention layer, generating the output representation \(\rm{Y}=(y_{1},y_{2},\ldots,y_{n})\). Then, \(\rm{Y}=(y_{1},y_{2},\ldots,y_{n})\) undergoes processing in the feedforward neural network, resulting in the output representation ERNIE \(\rm{Z}=(z_{1},z_{2},\ldots,z_{n})\). Figure 3: Schematic diagram of the ERNIE model structure. The Transformer encoder [35] comprises four essential components: word embedding and positional encoding, attention mechanism, layer normalization and residual connections, and feedforward neural network. Word embedding and positional encoding are integral components of the Transformer encoder. These features encode positional information for each word in the text, enabling the Transformer to capture the relevance and temporal characteristics of individual words. This, in turn, contributes to improving the overall performance of the model. \[X=E(X)P,X\in R^{b\times l\times d} \tag{1}\]  
In Equation (1), \(E\) is a matrix with rows representing the number of words. \(P\) represents the positional encoding vector, B corresponds to the number of text inputs in a batch, L represents the length of each text (seq_len), and D denotes the dimension of each word embedded into the matrix (embed_dim).