Label-Aware Hyperbolic Embeddings for Fine-grained Emotion Classification (2023)

Abstract: Fine-grained emotion classification (FEC) is a challenging task. Specifically, FEC needs to handle subtle nuance between labels, which can be complex and confusing. Most existing models only address text classification problem in the euclidean space, which we believe may not be the optimal solution as labels of close semantic (e.g., afraid and terrified) may not be differentiated in such space, which harms the performance. In this paper, we propose HypEmo, a novel framework that can integrate hyperbolic embeddings to improve the FEC task. First, we learn label embeddings in the hyperbolic space to better capture their hierarchical structure, and then our model projects contextualized representations to the hyperbolic space to compute the distance between samples and labels. Experimental results show that incorporating such distance to weight cross entropy loss substantially improve the performance on two benchmark datasets, with around 3% improvement compared to previous state-of-the-art, and could even improve up to 8.6% when the labels are hard to distinguish. Code is available at https://github.com/dinobby/HypEmo .

## introduction fine-grained classification is a challenging yet important task that involves differentiating subtle distinctions in a label set. for instance, in image classification, classifying cars, planes, and other vehicles is coarse-grained classification, whereas distinguishing models from cars is fine-grained classification. in nlp, sentiment analysis, which attempts to classify positive/negative sentiments, is an example of coarse-grained text classification. human emotions, however, exhibit more complexity. for example, the six type of basic emotion @xcite include happiness, sadness, fear, disgust, anger, and surprise, and show finer distinctions of positive and negative classes. moreover, figure 1 : properties of hyperbolic space. the hyperbolic distance between two points grows dramatically as they come close to the border, converse to the euclidean distance. moreover, the hyperbolic distance is an analog of tree distance, which is d(b, c) ≈ d(b, a) + d(a, c). existing text classification methods often use pre-trained language models such as bert @xcite to generate a contextualized represen-tation, and fine-tune them primarily in euclidean space for downstream tasks. however, such a space is limited, as some confusing pairs have nearly identical meanings (e.g., furious and angry), and forcing them to be separated in the latent space may harm the performance by overfitting on training data. the complex nature of emotions can be expressed in a hierarchical way, consisting of three levels @xcite . for instance, joy, contentment, and pleasure are primary, secondary, and tertiary emotions, respectively. meanwhile, learning embeddings in hyperbolic space is becoming more popular due to its superior ability to capture hierarchical information @xcite @xcite . figure 1 demonstrates a tree embedded to hyperbolic space. tree nodes closer to the root are embedded near the origin, and nodes closer to leaves are placed closer to the boundary. the main merit of this space is that as the distance from the origin rises, the amount of space in hyperbolic space grows exponentially @xcite @xcite . intuitively, tree-like structures also expand the number of nodes as the distance increases from the root, which is consistent with the mathematical basis of hyperbolic geometry. this is also reflected in the hyperbolic distance (eq. 6), which resembles the distance between two nodes in a tree. in this work, we propose hypemo, which integrates label embedding trained in hyperbolic space with a roberta model fine-tuned in euclidean space. specifically, we first learn hierarchicalaware label embeddings in hyperbolic space, and then project the representation output by roberta onto the same space to derive the distance between a text representation and its corresponding label. this distance is then used to weight standard cross entropy loss, making the projection of text representation as close to its label as possible, in hyperbolic space. results on two challenging datasets, goemotions @xcite and @xcite , demonstrate the superiority of the proposed model. also, we find that hypemo performs best when the label structure is more complex, or the inter-class relationship is more ambiguous. to sum up, the contributions of this paper are threefold: ## methodology in this section, we describe the proposed hypemo in detail. fig. 2 illustrates its workflow. ## conclusion we propose hypemo, a novel framework that includes a label-aware hyperbolic weighted loss to improve fec task performance. by jointly learning the representations in euclidean and hyperbolic space, we leverage hybrid settings that combine the power of large-scale pre-trained language models and the mathematical characteristics of hyperbolic space to capture the hierarchical property of classes and the nuanced differences between them. with this design, the proposed method achieves stateof-the-art results in terms of weighted f1 on the ge and ed benchmark datasets. we show that the proposed model works even better when the labels are difficult to differentiate. moreover, hypemo outperforms methods that utilize data augmentation while being more efficient.