# Sentiment Analysis of Comment Texts on Online Courses Based on Hierarchical Attention Mechanism
## 3 Model Building
### Model Construction Process  
In order to obtain the local sentiment features of reviews, we use CNN to obtain the local features of sentences (i.e., \(H_{\text{C}}=\left[h_{c}^{1},\ h_{c}^{2},\ \cdots,\ h_{t}^{m}\right]\epsilon\)) as shown in Figure 1. Then, to further extract the hidden features of the text, LSTM is used to extract the hidden information of the comment text. Finally, the context hidden state (i.e., \(H_{L}=\left[h_{l}^{1},\ h_{l}^{2},\ \cdots,\ h_{t}^{n}\right]\epsilon \mathbb{R}^{n\times d_{h}}\)) is extracted by LSTM. After that, a gate mechanism is used to screen the important sentiment information of \(H_{\text{C}}\) and \(H_{L}\), as follows. First, average pooling needs to be performed for the hidden state of the sentence \(H_{L}\) extracted by LSTM and the local emotion information \(H_{\text{C}}\), as shown in Formula (1) and Formula (2), respectively. The LSTM hidden layer vector representation \(h_{g}^{L}\in\mathbb{R}^{d_{h}}\) is obtained. Additionally, a local aggregate information vector representation \(h_{g}^{C}\in\mathbb{R}^{d_{c}}\) is obtained. \[h_{g}^{L}=\sum_{i=1}^{n}H_{L}/n \tag{1}\]  
\[h_{g}^{C}=\sum_{i=1}^{m}H_{c}/m \tag{2}\]  
Then the local emotion information extracted from CNN needs to be filtered through the gate mechanism by using \(h_{g}^{L}\), and the specific gate mechanism calculations are shown in Formulas (3)-(5). \[T_{\text{C}}=relu\left(H_{\text{C}}W_{\text{C}}+W_{g}h_{g}^{L}\times w_{g}\right) \tag{3}\]  
\[E^{i}=tanh\left(H_{\text{C}}^{i}W_{\text{E}}\right) \tag{4}\]  
\[G_{\text{C}}^{i}=E^{i}T_{\text{C}} \tag{5}\]  
In these calculations, \(W_{\text{C}}\in\mathbb{R}^{d_{c}\times d_{c}},\ \ W_{g}\in\mathbb{R}^{m\times d_{h}},\ W_{g}\in \mathbb{R}^{d_{c}}\) are parameter vectors; \(relu,\ tanh\) are activation functions; and \(W_{\text{E}}\in\mathbb{R}^{d_{c}\times m}\), \(G_{\text{C}}^{i}\in\mathbb{R}^{d_{c}}\) are global features. A selec  
Figure 1: Overall model structure. tive representation of the gate mechanism, \(G_{C}\in\mathbb{R}^{m\times d_{c}}\), can be obtained after expressing the chosen \(i\) vector of \(H_{c}\) through the gate mechanism. \(G_{L}\in\mathbb{R}^{n\times d_{h}}\) can be obtained in the same way. After \(G_{C}\) is obtained, information is aggregated by the attention mechanism, as shown in Formulas (6) and (7). \[\alpha=softmax\ (G_{C}w_{\alpha}+H_{C}w_{C}) \tag{6}\]  
\[h_{c}=\sum_{i=1}^{m}\alpha^{i}G_{c}^{i} \tag{7}\]  
\(w_{\alpha}\in\mathbb{R}^{d_{c}}\), \(w_{C}\in\mathbb{R}^{d_{c}}\) are parameter vectors, and \(H_{C}\) is the feature information extracted by the original CNN. After obtaining the selection information \(G\) of the gate mechanism, the original information \(H_{C}\) is further added when the attention mechanism coefficients are weighted to avoid the loss of original information. Finally, the vector \(h_{C}\in\mathbb{R}^{d_{h}}\) is weighted by Formula (7). The single-layer attention mechanism can only focus on the strong sentiment words in the sentiment expression while ignoring the most important words in sentiment analysis. In order to highlight the importance of different words in sentiment analysis, this study proposes to use a multi-layer attention mechanism to focus on the importance of different words in sentences; therefore, weighted by the first layer of attention mechanism \(h_{c}\). After that, to improve the accuracy of sentiment analysis by using text information, the information of \(G_{L}\) is further weighted by \(h_{c}\), as shown in Formulas (8)-(10). \[\gamma\ \left(h_{L}^{i},\ h_{c}\right)=tanh\ \left(h_{L}^{i}W_{L}h_{c}^{T}\right) \tag{8}\]  
\[h_{g}^{L}=\sum_{i=1}^{n}H_{L}/n \tag{1}\]0  
\[h_{g}^{L}=\sum_{i=1}^{n}H_{L}/n \tag{1}\]1  
\(h_{L}^{i}\in\mathbb{R}^{d_{h}}\) is the \(i\) vector of \(G_{L},\ tanh\) is the activation function, \(W_{L}\in\mathbb{R}^{d_{h}\times d_{C}}\) is parameter matrix, and \(h_{c}^{T}\) is the transpose of the \(h_{c}\) vector.