## 3 Materials and Methods
### Valence-Arousal Analysis  
This section introduces the features used in the Valence-Arousal analysis step. It can be divided into three parts: (1) The part-of-speech (POS) labels of Chinese morphemes. (2) Morpheme classification according to the National Taiwan University Sentiment Dictionary (NTUSD) [21] (3) Transform the morphemes into concept tags using the E-HowNet [22] and calculate the similarity to obtain the Valence and Arousal values as our features. Due to the characteristics of Chinese writing conventions, spaces are not used to separate words as they are in English. So, an important step in Chinese NLP is to first conduct tokenization. To do that, this study employed Multi-Objective NER POS Annotator (MONPA) [23], which was trained on (Traditional) Chinese corpus for this purpose. It provided segmentation, POS tags, and named entity recognition (NER) functionality specifically for Traditional Chinese. We first acquired the POS of each Chinese word in the dataset through the MONPA system, and supplied the POS features to the proposed model structure. Then, we generatd 50-dimension vectors as the representation for word sentiment prediction. Figure 3: Overall workflow of the research. Besides adding the POS feature vectors of each word, we employed a sentiment dictionary to further empower our model. The NTUSD is a sentiment dictionary that has been established by the Natural Language Processing Laboratory from National Taiwan University, and it includes 2810 positive and 8276 negative Chinese words. For example, positive words include \"harmony\", \"confidence\", \"contentment\", etc. ; negative words include \"despiteful\", \"insecurity\", \"unrealistic\", etc. We used the proposed model to preform binary classification by introducing NTUSD and determined the Valence and Arousal values of words. Specifically, when a word appeared in the positive dictionary, we set \"1\" as its category; when it appeared in the negative dictionary, we set \"\(-\)1\" as its category. Moreover, we calculated the cosine similarity and sorted the values for those unknown words that were not included in NTUSD. We selected the top five words in NTUSD that were most similar to the unknown words, which allowed us to complete the word classification task based on the positive or negative labels in NTUSD. We employed this word classification method to capture more sentimental features. Additionally, the E-HowNet, which is built upon the HowNet knowledge, was applied to convert the meaning of a word into more abstract concepts and to systematically express the word with a concept. For example, the concept of \"indignant\" is the same as \"resentful\". Additionally, the concept of \"faith\" is identical to \"trust\". Furthermore, we calculated the average of the Valence and Arousal values for the words that shared the same concept in order to obtain meaning features of the words. If there were no matching concepts that could be found, cosine similarity was computed to discover the closest concept, and the values of Valence and Arousal were represented by their mean value. In this way, two-dimensional features of abstractive semantics could be acquired. Finally, embeddings from Bidirectional Encoder Representations from Transformers (BERT) [24] were incorporated with the features mentioned above, which were able to depict more complex sentiments of different words in the context. In the end, there were a total of 821 dimensions in the final representative vectors, which were a combination of BERT embedding of the vocabularies, one by one, and associated with the POS feature vectors, sentimental features in NTUSD, and E-HowNet. Bi-directional Long Short-Term Memory can not only learn dependency within a greater distance, but can also better incorporate bi-directional information through the combination of two LSTMs [25], one from front to back and the other in the opposite direction. Furthermore, the Attention layer is capable of learning the different weights of each part of the input and extracting more crucial information [26].