## 3 Proposed Method
### Review Rating Prediction
#### 3.5.1 Architecture of HAN  
HAN [24] model comprises several elements, namely word attention, sentence attention layer, sentence encoder and word encoder. This network considers an extracted feature \(D_{g}\) as input for Review Rating Prediction process.  
_(a) Word encoder:_ The input \(D_{g}\) was embedded to vectors based on the embedding matrix \(F\). In addition, bidirectional Gated Recurrent Unit (GRU) was applied to obtain word annotations through data summarizing from both the directions for signal. However, bidirectional GRU includes forward GRU, which reads features from first to last, whereas backward GRU reads features from last to first.  
\[A =FD_{g}\cdot;D_{g}\in[1,\ldots,Y] \tag{31}\] \[\vec{m} =\overrightarrow{GRU}(A),D_{g}\in[1,Y]\] (32) \[\vec{m} =\overleftarrow{GRU}(A),D_{g}\in[Y,1] \tag{33}\]  
Here, annotation was attained for given data augmentation output \(D_{g}\) by combining forward hidden state \(\overrightarrow{m}\) and backward hidden state \(\overrightarrow{m}\) so that \(m=\left[\overrightarrow{m},\overrightarrow{m}\right]\).  
_(b) Word attention:_ Every word does not uniformly contribute to sentence meaning representation. Conversely, attention component was devised to extracts words, which are important in a sentence. The representation of instructive features was combined to generate sentence vector.  
\[z_{\text{p}}=\tanh(F\cdot m+o) \tag{34}\]  
\[\rho=\frac{\exp\left(z_{\text{p}}^{J}z_{E}\right)}{\sum_{Q}\exp\left(z_{\text{ p}}^{J}z_{E}\right)} \tag{35}\]  
\[E=\rho\cdot m \tag{36}\]Originally, word annotation \(m\) was subjected to one-layer MLP to acquire \(z_{\text{p}}\) as the hidden representation of \(m\); after that, significance of word as the correspondence of \(z_{\text{p}}\) was computed with word-level context vector \(z_{\text{E}}\) and normalized weight \(\rho\) was attained based on softmax function. Afterward, the sentence vector \(E\) was estimated using a weighted sum of word annotation by considering weights. Consequently, the context vector \(z_{\text{E}}\) was observed as the high-level representation of the instructive word. The word context vector was randomly initialized and learned mutually in the training process. The architectural model of the HAN classifier is depicted in Figure 3.  
**(c) Sentence encoder.** The document vector was derived and the bidirectional GRU was employed for encoding the sentences with sentence vector.  
\[\overrightarrow{m_{f}}=\overrightarrow{GRU}(E) \tag{37}\]  
\[\overleftarrow{m_{f}}=\overleftarrow{GRU}(E) \tag{38}\]  
Figure 3: An illustration of Hierarchical Attention Network (HAN).  
The sentence annotation was attained by concatenating \(\overrightarrow{m_{f}}\) and \(\overleftarrow{m_{f}}\). Here, \(m_{f}\big{[}m=\overrightarrow{m_{f}},\overleftarrow{m_{f}}\big{]}\) summarized nearby sentences about the sentences.  
_(d) Sentence attention:_ In sentence attention, a sentence-level context vector \(L\) is employed to identify sentence significance.  
\[z_{f}=\tanh\Big{(}F_{t}m_{f}+o_{t}\Big{)} \tag{39}\]  
\[\eta_{f}=\frac{\exp\Big{(}z_{f}^{l}V\Big{)}}{\sum_{f}\exp\Big{(}z_{f}^{l}V \Big{)}} \tag{40}\]  
\[Z=\sum_{f}\eta_{f}\cdot m_{f} \tag{41}\]  
where \(Z\) signifies document vector, which includes text information in a document. The final review rating predicted output of the HAN classifier is denoted as \(R_{t}\).