# Hierarchical Transformer Network for Utterance-Level Emotion Recognition
## 1 Introduction  
Sentiment analysis, considered one of the most important methods for analyzing real-world communication, is a kind of classification task for extracting emotion from language. It can help us progress in many fields. For example, in the field of data mining, we can research the financial volatility with sentiment analysis, and it can also drive the development of human-computer interactions. In this paper, we consider one of the tasks in this research direction, utterance-level emotion recognition (ULER) [1]. In ULER, an utterance [2] is a unit of a dialog bounded by breathes or pauses, and our goal is to tag each utterance with the indicated emotion (e.g., happy, sad, or angry). Traditional sentiment analysis methods are confined to analyzing only a single sentence or document, regardless of its surrounding information. However, in the field of ULER, contextual information is indispensable in emotional discrimination. When we analyze the emotion of an utterance, the contextual information represents the information from the front and back utterances. For example, in Figure 1, the utterance \"Yes, I agree with this point.\" can deliver different emotions in different contexts. To identify a speaker's emotion precisely, [PERSON] et al. [3] proposed contextual representations for prediction with a recurrent neural network (RNN), where each utterance is represented by a feature vector extractedby convolutional neural networks (CNN) at an earlier stage. Similarly, [PERSON] et al. [4] proposed a hierarchical gated recurrent unit (HiGRU) framework with a lower-level GRU to model the word-level inputs and an upper-level GRU to capture the contexts of utterance-level embeddings. Theoretically, RNNs such as long short-term memory (LSTM) and gated recurrent units (GRUs) should propagate long-term contextual information. However, in practice, this is not always the case [5]. In cases where the input sequence is long, RNNs may experience an exploding gradient or vanishing gradient. Unlike traditional text classification problems, in the field of ULER, there are a limited number of datasets, and most datasets contain inadequate conversations. This issue limits the possibility of obtaining larger models for this task. To solve this issue, [PERSON] et al. [6] proposed a knowledge enriched transformer (KET) to effectively incorporate contextual information and external knowledge bases, but this model structure is complex, and the running speed is not high. [PERSON] et al. [7] proposed pretraining a context-dependent encoder (CoDE) for ULER by learning from unlabeled conversation data to address the aforementioned challenge, but the model did not perform better in the word-level embedding phase. [PERSON] and [PERSON] [8] proposed two different bidirectional encoder representation from transformers (BERT) [9] models to deal with two different datasets and performed pretraining with different data sets. They got excellent results in the EmotionX Challenge 2019. However, different pre training methods are used in the two datasets, so the universality of the approach may be weak. In this task, we propose a hierarchical transformer framework to solve the above issues. First, we use a transformer [10] to model the word-level input and capture the contexts of utterance-level embeddings, which has been shown to be a powerful representation learning model in many NLP tasks and can exploit contextual information more efficiently than RNNs and CNNs. Second, for the data scarcity issue, we use a pretrained language model BERT [9] as the lower-level transformer, which is equivalent to introducing external data into the model and helps our model obtain better utterance embedding. Third, the same utterance can deliver different emotions in the same context. For example, in Figure 2, the utterance \"Yes, I agree. I think so, too.\" can deliver different emotions, joy, and sadness. A study also showed that speakers tend to mirror their counterparts to build rapport during the course of a dialogue [11].