EmoTransKG: An Innovative Emotion Knowledge Graph to Reveal Emotion Transformation (2024)

Abstract: This paper introduces EmoTransKG, an innovative Emotion Knowledge Graph (EKG) that establishes connections and transformations between emotions across diverse opentextual events. Compared to existing EKGs, which primarily focus on linking emotion keywords to related terms or on assigning sentiment dimension ratings to emotion words by humans, EmoTransKG aims to represent the general knowledge involved in emotion transformations. Specifically, in conversations, successive emotions expressed by a single speaker are temporally considered as the head and tail entities, with open-text utterances (events) occurring between them representing the relation. To explore the knowledge of emotion transformations described in EmoTransKG, we develop a Transformer-based translational model called EmoTransNet, which predictively trains tail entities by interpreting the relation as an operation that transforms the source emotion into the target emotion. Particularly, our designed EmoTransNet serves as a plug-in module that seamlessly integrates with any conversational emotion recognition (CER) models for emotion retrofitting. Experimental results on two CER datasets demonstrate that the incorporation of EmoTransNet with baseline models results in substantial improvements, and the qualitative visualization of entities and relations clearly clarify their unique roles in emotion transformations. These experiments confirm the quality and effectiveness of EmoTransKG.1

heading: 1 Introduction
text: In recent years, Knowledge Graphs (KGs)2, such as WordNet (Fellbaum, 1998), Google KG (Singhal, 2012), or ConceptNet 5.5 (Speer et al., 2017),
*Corresponding authors. 1Available at https://github.com/XP-ZHA/EmoTransKG 2In 2012, Google introduced the “Knowledge Graph”, emphasizing its structure around vertices and edges. In the context of knowledge graphs, any knowledge resource that can be converted into such a structure is viewed as a knowledge graph.
have emerged as influential repositories and conveyors of knowledge, with a growing expectation to endow computers with human-like cognitive abilities by providing a wealth of world knowledge, including general commonsense knowledge (Liu and Singh, 2004; Matuszek et al., 2006) and emotional knowledge (Bradley and Lang, 1999; Stevenson et al., 2007). Unlike commonsense knowledge, emotions are abstract concepts that resist direct quantification, characterized by fuzzy boundaries and significant variation in personal expression and experience, making the construction of the emotion knowledge graph (EKG) an inherent challenge.
Emotions and sentiments are pervasive in text, which refers to written language and transcriptions of communication. Currently, the development of EKGs relies heavily on these textual sources. In written language, where words serve as the smallest utterances of meaning in a language, many projects strive to rate word lexicons3 (Bradley and Lang,
3The word lexicon can be converted into an EKG by linking words and their annotations through the use of the RelatedTo relation.
12098
1999; Warriner et al., 2013; Mohammad, 2018a) in a fine-grained dimensional space of emotion, such as the three-dimensional VAD (Valence, Arousal, and Dominance) space (Osgood et al., 1957; Russell, 2003), arguing that individual emotions can be effectively conveyed through word meanings. However, word-based EKGs have limitations, including inconsistent annotations from different annotators, challenges associated with fixed granularity, and rating scales that may inadequately capture the intended sentiment. Additionally, there are textbased EKGs (Gill et al., 2008) that involve semantic analysis of context, going beyond simple word mapping. One such example is COMET (Bosselut et al., 2019), which expresses individual intentions and reactions toward textual events. Transcriptions of communication are mainly used for tasks related to emotion recognition or analysis, such as Conversational Emotion Recognition (CER) (Zhang et al., 2019), however, the study of how to make them work in EKGs is still open. Furthermore, these efforts concentrate on the expression of emotions rather than their transformations or connections.
Emotion transformation, as outlined by Greenberg (2002), is a process of changing emotions with emotions only through new experiences. Natural language emotion transformation tasks aim to rephrase input sentences to satisfy a given affective label. Some approaches (Helbig et al., 2020) use word-emotion lexicons, like WordNet (Fellbaum, 1998), to reverse the sentiment polarity of the text through word substitution techniques. Recent studies (Luo et al., 2019; Shen et al., 2017; Li et al., 2018) suggest that deep learning approaches, especially encoder-decoder models, have the potential to generate fluent and natural language. Nonetheless, the resulting text often lacks contextual consistency due to the absence of parallel data for supervision. Essentially, these approaches focus on altering the emotions expressed in the text, rather than exploring emotion transformation and its fundamental patterns and regularities. Access to emotion transformation holds benefits for a number of fields, including psychology (e.g., developing interventions for emotion regulation), natural language processing (e.g., understanding the interplay of language and emotion to enhance human-computer interactions), and neuroscience (e.g., understanding the dynamic mapping of neural circuitry underlying emotion experience).
In this paper, we describe how we construct and evaluate EmoTransKG, an emotion knowledge
graph designed to address the aforementioned gap in emotion transformation using communication transcripts. Specifically, we collect real-world conversation segments in which a speaker delivers initial and final utterances (referred to as "events") that are manually annotated with emotion keywords (e.g., happy, sad, and angry). For each segment, we define the emotions annotated in the one utterance and its successive one as head and tail entities respectively, and all textual utterances (a.k.a. emotional events (Scherer, 1993)) within them as a relation, resulting in the creation of an emotion knowledge triple. This construction is inspired by the cognitive theory proposed by Ortony et al. (1988), which views emotions as reactions to events. An example of converting a conversation segment into an EmoTransKG triple is illustrated in Figure 1. To evaluate EmoTransKG and analyze the underlying patterns and regularities in emotion transformation, we introduce EmoTransNet, a Transformer-based translational model that learns distributional representations of emotion knowledge triples. We then employ EmoTransNet for emotion retrofitting in CER to validate the effectiveness of EmoTransKG and the learned representations for visual analysis to uncover the impact of the head entity and the relation on the tail entity.
The primary contributions of this work can be summarized as follows:
• We propose a brand new emotion knowledge graph called EmoTransKG, which establishes connections among emotion entities and by defining unstructured, open-text event sequences as their relations. To the best of our knowledge, we are the first to construct emotion relations in such a knowledge graph framework.
• We introduce EmoTransNet, a Transformerbased translational model that captures the complex knowledge of emotion transformations embedded in the triples of EmoTransKG and the relatedness among relations.
• We apply EmoTransNet to retrofit the predicted emotions in CER baseline models and to visually analyze emotion knowledge triples. Experiments conducted on two benchmarking collections demonstrate that emotion retrofitting consistently enhances the performance of the models, and that EmoTransKG successfully establishes meaningful transformations and connections between emotions.

heading: 2 Related Works

heading: 2.1 Emotion Knowledge Graph
text: In the current literature, EKGs typically rely on two different representations of emotion: one represents emotions as points in a multidimensional appraisal space, and the other relies on predefined emotion keywords (Calvo and D’Mello, 2010). The former approach involves developing lexicons (Bradley and Lang, 1999; Warriner et al., 2013; Mohammad, 2018a) that map words onto a psychologically-based affect space. However, these emotion words are rated in a context-free environment, whereas in reality they are gathered in a context-dependent textual setting. In contrast, EmoTransKG constructs emotion knowledge that is not explicitly stated in the text and requires consideration of the event context. The latter approach focuses on text-based EKGs (Scherer, 1993; Gill et al., 2008) that attempt to establish connections between text and emotion keywords. However, the success of existing text-based EKGs (Gill et al., 2008) is limited to texts conveying fear and joy. To circumvent this limitation, some commonsense knowledge graphs (Bosselut et al., 2019; Zhang et al., 2020) utilize well-defined logical relations instead of strict matching relations. In contrast to their definition of texts as entities and without addressing the connections between emotions, EmoTransKG considers texts as relations between emotions, representing general knowledge of emotion transformations. This approach lowers our expectations for text awareness. In addition, the EmoTransKG structure exhibits a unique relation pattern of multi-edge, while also displaying common relation patterns—antisymmetry, inversion, and composition (Sun et al., 2019)—observed in existing KGs.

heading: 2.2 KG Representation Learning
text: In recent years, there have been significant advances in learning KG representations (Wang et al., 2017) for KG inference, with the translational approach gaining considerable popularity. This approach interprets relations as transformations from head entities to tail entities. TransE (Bordes et al., 2013), a seminal model in this family, assesses the plausibility of the triple by computing the semantic correlations of the triple items in a continuous feature space. To capture more complex structures in KGs, several variants of TransE, such as TransH (Wang et al., 2014), TransR (Lin et al., 2015), and TransD (Ji et al., 2015), have been de-
veloped. We also note that some works employ deep learning models to learn representations for KGs and formulate non-linear scoring functions for plausibility assessment. Notable works in this area include ConvKB (Nguyen et al., 2018) and HypER (Balazevic et al., 2019), which use convolutional neural networks, KEGCN (Yu et al., 2021), which utilizes a graph neural network, and both COMET (Bosselut et al., 2019) and KAN (Dun et al., 2021) build on the Transformer framework. In light of these advancements, we propose a Transformer-based translational model called EmoTransNet in this work. Our model not only effectively captures emotion knowledge encoded in event contexts (relations), but also identifies potential correlations among items within the triple.

heading: 2.3 CER Incorporating KG
text: CER has emerged as a crucial task in emotion recognition and interaction, with the goal of accurately predicting the emotion expressed by a speaker with each utterance or event in a conversation. Recent research in CER has focused on integrating external knowledge to enhance the understanding of conversational context. For instance, COSMIC (Ghosal et al., 2020), SKAIG (Li et al., 2021), and CauAIN (Zhao et al., 2022) integrate the commonsense knowledge graph COMET to strengthen the logic of contextual interactions. Similarly, KET (Zhong et al., 2019) introduces the emotion lexicon NRC VAD (Mohammad, 2018b) to enrich event representations. These efforts aim to enhance the perception of language and comprehension of event interactions, distinguishing them from the function of our EmoTransKG in CER; EmoTransKG aims to improve the transformation of the speaker’s emotions during a conversation.

heading: 3 Methodology
text: In this section, we first outline the construction process of EmoTransKG and its characteristics (§3.1). Next, we present the EmoTransNet model for training the EmoTransKG base (§3.2). Finally, we apply the EmoTransNet for emotion retrofitting in CER (§3.3).

heading: 3.1 Construction of EmoTransKG
text: EmoTransKG is an EKG designed to construct emotion transformations and connections by extracting segments from communication transcripts. In particular, in a conversation consisting of a sequence of N utterances/events U =
{u1, u2, . . . , uN} paired with corresponding emotion labels {y1, y2, . . . , yN} ∈ S, where S is a predefined set of basic emotions consisting of emotion keywords, a segment denoted as Ui−j = {ui, ui+1, . . . , uj} is extracted, such that i < j, along with two emotion labels {yi, yj}. It is essential to ensure consistent speaker identities for ui and uj , while distinguishing them from the rest of the events U(i+1)−(j−1). This one-hop link structure within the same speaker facilitates stronger connections between emotions, as the psychological interactions between events are locally effective, as claimed by Shen et al. (2021) and Li et al. (2021). Consequently, an EmoTransKG triple is created, where the head entity is yi, the relation is Ui−j , and the tail entity is yj , or succinctly expressed as (h = yi, r = Ui−j , t = yj) (Ref. Figure 1).
Notation. We define Xh ∈ S as the head entity, a sequence of textual events Xr = {xr1, xr2, . . . , xr|r|} as the relation, and Xt ∈ S as the tail entity in the EmoTransKG triple. Clearly, in this definition, the EmoTransKG triple is instantiated with two symbolic entities and a natural language relation.
Relation Pattern. EmoTransKG possesses a unique graph structure characterized by a closed class of entities and an open class of relations, setting it apart from other KGs. The relations in EmoTransKG are temporally directed and linguistically irreversible, align with certain logical relations, such as Causes and MadeOf in COMET (Bosselut et al., 2019), while differing from certain semantic relations, such as RelatedTo and SimilarTo in ConceptNet 5.5 (Speer et al., 2017). Therefore, it is essential to discuss the relation patterns in EmoTransKG:
• Antisymmetry: As the relation in EmoTransKG is directed and irreversible, for any given triples (h, r, t) and (t, r, h), it holds that:
(h, r, t) ∧ (t, r, h) ⇒ h = t • Inversion: EmoTransKG forms a strongly
connected graph, where for any given triple (h, r1, t), there exist a relation r2 such that:
(h, r1, t) ⇒ (t, r2, h) • Composition: EmoTransKG is a strongly
connected graph, with each node forming a closed loop with its edges. For any given triples (h, r1, t1) and (t1, r2, t2), there exist a relation r3 such that:
(h, r1, t1) ∧ (t1, r2, t2) ⇒ (h, r3, t2)
• Multi-edge: In graph theory, multi-edge refers to two or more edges incident to the same pair of vertices. In EmoTransKG, for any given entities h and t, we have
(h, r1, t) ∧ (h, r2, t) ⇒ r1 ̸= r2 A structure with such form is a multi-edge relation pattern, and its definition is first proposed and formalized in the domain of KGs.

heading: 3.2 EmoTransNet: Training EmoTransKG
text: Having created EmoTransKG, our goal is to study the influence of initial emotion (head entity) and events (relation) on future emotion (tail entity). To achieve this goal, we introduce EmoTransNet, a Transformer-based translational model designed to take the head entity Xh and its associated relation Xr as input and learn to generate the tail entity Xt as output, i.e., predicting t in (h, r, ?). The proposed EmoTransNet model is depicted in Figure 2.
Head Entity Representation. We represent symbolic head entities using one-hot vectors in the space R|S|. In order to enhance their expressive power, we define and use two fully connected layers to project the entity vectors into a higher dimensional feature space as follows:
hh = W h2 (W h 1 X h + bh1) + b h 2 . (1)
Here, W h1 ∈ R|r|×dh , W h2 ∈ Rdh×d, bh1 ∈ Rdh , and bh1 ∈ Rd are trainable parameters. The resulting output, hh ∈ Rd, is considered to be the representation of the head entity.
Relation Representation. To reduce computation in extracting relation representations, we employ a two-step feature extraction scheme to account for factors such as the language length of the relation. Initially, we extract the event-level feature representation hri ∈ Rd for each event xri using a pretrained language model. Although the event itself is independent of the language model, we use the RoBERTa model (Liu et al., 2019) in this work.
Upon obtaining the event representations {hr1, hr2, . . . , hr|r|} from RoBERTa, we proceed to capture the relation semantics involving these events. To achieve this, we prepend a special token, [CLS], which serves as an aggregator for the relation information, at the beginning of the event representations. This results in the creation of the input sequence hr = {hr[CLS], hr1, hr2, ..., hr|r|} ∈ R(|r|+1)×d. In order to account for the directed nature of the relation, we introduce position embeddings pt ∈ R(|r|+1)×d encoded by sine and cosine functions into the input sequence:
hr,0 = hr + pt. (2)
Next, we employ a standard Transformer architecture (Vaswani et al., 2017) with L layers to encode the input sequence hr,0 ∈ R(|r|+1)×d. The l-th Transformer block can be formulated as follows:
g̃r,l = MULTIATTN(hr,l−1), (3) gr,l = LAYERNORM(g̃r,l + hr,l−1), (4)
h̃r,l = FFN(gr,l), (5)
hr,l = LAYERNORM(h̃r,l + gr,l), (6)
where MULTIATTN is a multi-head self-attention mechanism, LAYERNORM is a layer normalization operation, and FFN is a two-layer feed-forward neural network with ReLU activation. After propagating through L layers, the hidden state hr,L[CLS] ∈ Rd of the [CLS] token from the last layer is adopted as the representation of the relation.
Tail Entity Representation. The head entity representation hh and the relation representation hr,L[CLS]
are concatenated and fed into a fully connected layer to obtain the representation of the tail entity:
ht = W t1[h h∥hr,L[CLS]] + b t 1. (7)
Here, ∥ denotes the concatenation operation. W t1 ∈ R2d×d and bt1 ∈ Rd are learnable parameters. Loss Function. Finally, we utilize a fully connected layer with softmax activation as the classifier to predict the tail entity:
Ȳ = softmax(W t2h t + bt2), (8)
where W t2 ∈ Rd×|S| and bt2 ∈ R|S|. The crossentropy loss utilized to train the EmoTransNet model is calculated on all tail entities by:
L = − 1 M
M∑
i=1
|S|∑
e=1
yei log(Ȳ e i ), (9)
where M is the number of triples in EmoTransKG, yei is the one-hot vector denoting the tail entity of triple i, and e is the dimension of each entity.

heading: 3.3 CER with EmoTransNet
text: One way to assess the plausibility and effectiveness of KGs is to examine the correlation and consistency between entities and relations. The CER task serves as a benchmark for evaluating emotion propagation and transformation, testing whether a method can accurately capture emotion interactions in a conversational context. In this study, we employ the CER task as an automatic evaluation metric for EmoTransKG through EmoTransNet.
Retrofitting, as redefined by Faruqui et al. (2015), is a process of adjusting word embeddings using a knowledge graph. Generalizing this concept, we utilize EmoTransKG to adjust the predicted emotion confidence vectors produced by the CER model, which we refer to as “emotion retrofitting.” Figure 3 illustrates the integration of EmoTransNet with the CER model, wherein EmoTransNet’s parameters are frozen. The CER framework comprises three fundamental modules: Feature Extraction (FE) for encoding the semantic embeddings of each utterance, Conversational Context Modeling (CCM) for capturing interactions between utterance units or nodes using a recurrent- or graphbased model, and Emotion Recognition (ER) for predicting the emotion label of each utterance. This can be expressed as follows:
Ỹ = ER(CCM(FE(U))), (10)
where Ỹi represents the predicted emotion confidence vector for utterance ui.
The emotion retrofitting function can be represented as follows:-
Ŷj = γỸj+(1−γ)EmoTransNet(Yi, Ui−j), (11)
where γ is a hyper-parameter that controls the preservation of emotion confidence in the predicted vector space. Note that, (1) the turns i and j correspond to the same speaker; (2) Yi = One-Hot(Ỹi), where One-Hot(·) is a one-hot assignment operation: the predicted emotion confidence vector of the first utterance of each speaker is not adjusted; and (3) Yi = One-Hot(Ŷi): the adjusted emotion confidence vector Ŷi will replace the original Ỹi as the input of EmoTransNet. This replacement is made because Ŷi is more robust than Ỹi, even though the effect of this robustness on future emotions decays exponentially with sequence turns. The emotion retrofitting described above is a plug-in module; it can refine emotion confidence vectors predicted by any CER model, as the updates in Eq. 11 are agnostic to the CER model.
Another task of entity prediction in EmoTransKG is to predict h in (?, r, t). This facilitates the development of a bi-directional emotion retrofitting scheme similar to Bi-LSTM, although it is not currently operational. It is evident that our designed EmoTransNet is conceived as a single approach for extracting knowledge concerning emotion transformation within EmoTransKG.

heading: 4 Experiments

heading: 4.1 Experimental Setup
text: Dataset and Evaluation Metrics. EmoTransKG uses segments from conversation transcripts to create emotion knowledge triples. In this work, we use the IEMOCAP (Busso et al., 2008) and EmoryNLP (Zahiri and Choi, 2018) resources as conversation transcripts, but other conversation resources could have been used as well, since the ambiguity surrounding the definition of basic emotions. IEMOCAP is a collection of dyadic conversations that defines basic emotions as happy, sad, neutral, angry, excited, and frustrated, based on conversational content, facial expressions, and hand movements. EmoryNLP consists of textual transcripts from multi-speaker conversations in the TV show Friends and defines basic emotions as follows: positive: {joyful, peaceful, powerful}, negative: {scared, sad, mad}, neutral: {neutral}.
Table 1 presents the statistics of all datasets and their corresponding EKGs. Following recent CER works (Ghosal et al., 2020; Li et al., 2021), we use only the textual data from the above datasets and choose weighted-F1 as the evaluation metric for CER tasks. For both the creation of EmoTransKG and the training of EmoTransNet, the training set of each dataset is used. Additionally, to test the statistical significance of the performance improvement, we perform a paired t-test (Kim, 2015).
Baseline Models. Given that emotion retrofitting is a model-agnostic plug-in that facilitates the integration of various CER models, we select ten CER models to test whether EmoTransKG can improve their performance. Please refer to Appendix A for a comprehensive list of the baseline models and to Appendix B for implementation details.

heading: 4.2 Evaluation Results and Analysis
text: The overall results are showcased in Table 2, where “X+EmoTransKG” indicates the combination of model X with the emotion transformation knowledge obtained from our proposed EmoTransKG. EmoTransKG shows consistent improvement in performance with high confidence (p<0.05) on both the IEMOCAP and EmoryNLP datasets when evaluated within the same CER framework. These improvements on CER highlight the effectiveness of our EmoTransKG and the applicability of emotion retrofitting. In particular, the performance improvements achieved by EmoTransKG on models modeling intra-speaker dependencies demonstrate the breadth and depth of its understanding of emotion transformations. Implementing EmoTransKG on the external knowledge-introduced models also enhances their performance, emphasizing that the emotion transformations constructed by EmoTransKG differ from logical interactions and provide improved performance in emotion interactions from a new perspective.
Although our approach performs acceptably on EmoryNLP, it does not yield results as remarkable as those on IEMOCAP. Table 1 provides two reasons for this discrepancy. One reason is that
the ratio of the number of relations to the number of utterances in the IEMOCAP test set is nearly twice that of the EmoryNLP test set. This implies that our EmoTransNet model runs emotion retrofitting about twice as frequently in IEMOCAP as in EmoryNLP during testing. Additionally, consuming fewer utterances per relation (2.97<3.52) leads to more robust emotion transformations.

heading: 4.3 Visualization Analysis
text: To intuitively assess the semantic relatedness of relations and the impact of the head entity and the relation on the tail entity in EmoTransKG, we project the learned relation representations (hr,L[CLS]) and tail entity representations (ht) from EmoTransNet into a 2D space using t-SNE (Maaten and Hinton, 2008). The resulting visualizations are presented in Figures 4 and 5, where each color represents a different head or tail entity category that is assigned in the EmoTransKG triples.
As you can see in Figures 4(a) and 5(a), EmoTransNet invariably generates similar relation representations for EmoTransKG triples with identical head and tail entities, demonstrating their semantic affinity. However, while Figure 4(a) reflects the categories of both head and tail entities, Figure 5(a) only shows the categories of tail entities. The difference in the EmoryNLP-derived EmoTransKG can be attributed to the strong correlation between tail entities and relations, which masks the weak correlation with head entities. Figure 4(b) demonstrates the separability of tail entity representations based on head entities (e.g., region shown in the dashed red ellipse for the same tail entity "frustrated") and relations (e.g., region shown in the dashed blue ellipse for the same head entity "neutral"). Such a visualization suggests that the head entities determine the intra-class distribution of tail entities, and the relations determine the inter-class distribution. Similarly, Figure 5(b) shows a comparable distribution; however, with more blurred boundaries in the left figure and clearer boundaries in the right figure. This discrepancy indicates that, compared to the IEMOCAP-derived EmoTransKG, the head entity has less influence, while the relation has a stronger impact on the tail entity in the EmoryNLP-derived EmoTransKG. These separable distributions explain the effectiveness of our EmoTransNet in extracting emotional knowledge and the plausibility of emotion transformations within EmoTransKG. To further support the differences in visualizations
between the two EmoTransKGs, we provide their quantitative analyses in Appendix C.

heading: 4.4 Ablation study
text: We present the following ablations of the EmoTransKG construction, without considering the model structure and training techniques for EmoTransNet. Firstly, we investigate how varying the amount of relations available for the construction of EmoTransKG affects the diversity of emotion transformations. Secondly, in our initial notation of EmoTransKG, head entities are represented as separate onehot vectors, which requires EmoTransNet to learn emotion-specific semantic representations from scratch. As an ablation, we train a model by mapping head entities to natural language and initializing them with Word2vec embeddings (Mikolov et al., 2013) to incorporate knowledge from language, which we denote EmoTransKG-WE. We report the ablation results of COSMIC and DialogueGCN in Table 3.
We observe that increasing the number of relation facts can be increased to allow the model to learn a greater diversity of emotion transformation knowledge, thus improving its generalization ability. When using word embedding initialization, it has been noticed that there is inferior performance in CER models. One hypothesis is that this occurs because the inaccuracy of the provided Word2vec embeddings. To validate this hypothesis, we compute the cosine similarity between Word2vec em-
beddings of basic emotions on EmoryNLP, and find that the cosine similarity between Joyful and Sad is higher than that between Joyful and Powerful, consistent with Agrawal et al. (2018)’s observations. This result corroborates that word embeddings derived from textual corpora are inadequate for accurately capturing emotion semantics, presenting a significant challenge for word embedding learning. In contrast, the entity representations (hh) learned from EmoTransNet demonstrate that emotions with the same polarity exhibit higher cosine similarity compared to those with different polarities, indicating more effective semantic learning of emotions. In addition, we observe a steeper performance degradation in IEMOCAP due to the stronger influence of head entities on tail entities in the IEMOCAP-derived EmoTransKG, as revealed through our visualization analysis (§4.3).

heading: 4.5 Case Study
text: In Figure 6, we present two cases (#13 → #16 and #15 → #18) of emotion prediction, where our EmoTransNet yields accurate predictions while COSMIC falls short. In both conversation segments, the continuous emotions conveyed in the individuals’ first and last utterances are transferred due to the experienced events. Therefore, the model must consider both emotional inertia resulting from prior emotions and the perception of events.
Specifically, EmoTransNet assigns the emotion label "excited" to #16, despite events #14 and #15 conveying the positive semantic polarity "happy." This suggests that emotion transformation is influenced by the initial emotion (the head entity) rather than solely by the experienced events (the relation). Conversely, COSMIC’s prediction for #16 merely replicates or empathizes with the speaker M’s emotion. For the prediction of #18, despite its sequential modeling, COSMIC struggles to capture or comprehend historical information necessary for accurate prediction. This highlights COSMIC’s limitations in emotional interaction and contextual modeling, despite the presence of contextual utterances providing effective cues for emotion prediction. Our EmoTransNet, which adopts
an emotion transformation perspective, predicts future emotions based on initial emotions and experienced events. This approach mitigates potential misjudgments resulting from emotional contagion and from information loss during long-distance transmission.

heading: 5 Conclusion
text: In this paper, we present EmoTransKG: an emotion knowledge graph. We define basic emotions in conversation transcripts as emotion entities, and extract event sequences between two entities with the same participant as emotion relations. EmoTransKG is the first EKG that establishes emotion transformations and connections. We train an EmoTransNet model on EmoTransKG and utilize it for emotion retrofitting in CER, as well as to visualize the roles of entities and relations in emotion transformations, thereby demonstrating the plausibility and effectiveness of EmoTransKG. Currently, we are extending our approach to other conversational resources, such as DailyDialog (Li et al., 2017) and MELD (Poria et al., 2019), and fusing these EmoTransKGs to achieve high coverage of emotion concepts. We also investigate whether EmoTransKG can offer recommendations for the engineering of emotion-related word embeddings.
Limitations
We note several limitations of our work:
(1) Our EmoTransKG is limited by the fact that the definition of basic emotions is disputed, which poses a challenge for future efforts to extend and fuse emotion knowledge graphs. Although our work has not addressed this issue, our visualization results demonstrate basic emotions from different emotion theories can be well separated from each other. These findings provide valuable insights for advancing research in emotions and cognition.
(2) The diversity of relations requires a significant amount of available emotional transfer facts for successful editing, but the linguistic richness and diversity limit its coverage.
(3) We do not currently account for data imbalance in EmoTransKG. As shown in Figures 4(b) and 5(b), the EmoTransNet model, trained on the target variable (tail entity) with such a significantly skewed distribution, exhibits biased, generally leading to better performance for emotion triples where the head and tail entities are the same.

heading: Acknowledgements
text: This work was supported by National Natural Science Foundation of China (NSFC Grant No. 62076092).

heading: A Baseline Models
text: We use the following baseline models in our evaluation, and their corresponding codes and parameters are uploaded in the supplementary material:
Recurrence-based Models: DialogueRNN (Majumder et al., 2019), COSMIC (Ghosal et al., 2020), and BiERU (Li et al., 2022).
Graph-based Models: DialogueGCN (Ghosal et al., 2019), RGAT (Ishiwatari et al., 2020), SKAIG (Li et al., 2021), DAG-ERC (Shen et al., 2021), CauAIN (Zhao et al., 2022), SUNET (Song et al., 2023), and DualGATs (Zhang et al., 2023).
These models are chosen based on their strong performance and prominence in the CER field, as well as the availability of source code to ensure the reliability and reproducibility of results. All
models, except for BiERU, explicitly build intraspeaker dependencies to facilitate the comprehension of emotion propagations and connections of individual speakers in conversations. Additionally, COSMIC, SKAIG, and CauAIN incorporate external knowledge into their frameworks to enhance the logical interactions between utterances/events.
B Implement Details
This section details the hyper-parameters and optimal combinations selected. Specifically, we set dh to 300, d to 1024, and γ in Eq. 11 to 0.95. During the training phase of EmoTransNet, we utilize the Adam optimization algorithm (Kingma and Ba, 2014) with a batch size of 64, 100 epochs, a learning rate of 2e-5, a dropout rate of 0.5, and a weight decay of 3e-4. To balance performance and computational complexity, we set the number of Transformer blocks L to 6. Consistently, we use the RoBERTa (Liu et al., 2019) feature extractor to obtain utterance embeddings for the Feature Extraction module in all baseline models mentioned. For the re-implementations and emotion retrofitting experiments, we strictly adhere to the settings reported in the original papers. The experiments are conducted on a server equipped with a 3090 GPU card and the Ubuntu operating system version 20.04. The reported results are based on the average score obtained from 20 random runs on the test set.

heading: C Quantitative Analysis
text: In this section, we perform a quantitative analyses of emotion transformations within EmoTransKG. We examine the two following aspects:
(1) Impact of head entities on emotion transformations. Specifically, for each triple in the validation set, we randomly select a different emotion entity to replace its head entity, and then input the replaced head entity and its corresponding relation into EmoTransNet to assess the probability of correctly predicting the tail entity. The EmoryNLP dataset shows a significantly higher probability of correctly predicting the tail entity (85%) compared to the IEMOCAP dataset (62%). This result indicates that the head entities in the IEMOCAPderived EmoTransKG have a greater impact on predicting the tail entities than those in the EmoryNLP-derived EmoTransKG.
(2) Impact of relations on emotion transformations. We find that in the IEMOCAPderived EmoTransKG, 70% of the triples have the same head and tail entities. However, this proportion decreases to only 36% in the EmoryNLP-derived EmoTransKG. These results suggest that the relations derived from EmoryNLP are more likely to transform the head entity into a tail entity of a different category.
In conclusion, our quantitative analyses indicate that both head entities and relations in EmoTransKG triples have an impact on the emotional expressions of tail entities. Furthermore, the influence of head entities is relatively weaker in the EmoryNLP-derived EmoTransKG compared to the IEMOCAP-derived EmoTransKG, while the impact of relations is stronger. These findings are consistent with the visualization results (§4.3).

heading: Future Work
text: Limitations
We note several limitations of our work:
(1) Our EmoTransKG is limited by the fact that the definition of basic emotions is disputed, which poses a challenge for future efforts to extend and fuse emotion knowledge graphs. Although our work has not addressed this issue, our visualization results demonstrate basic emotions from different emotion theories can be well separated from each other. These findings provide valuable insights for advancing research in emotions and cognition. (2) The diversity of relations requires a significant amount of available emotional transfer facts for successful editing, but the linguistic richness and diversity limit its coverage. (3) We do not currently account for data imbalance in EmoTransKG. As shown in Figures 4(b) and 5(b), the EmoTransNet model, trained on the target variable (tail entity) with such a significantly skewed distribution, exhibits biased, generally leading to better performance for emotion triples where the head and tail entities are the same