Euphemism Detection by Transformers and Relational Graph Attention Network (2022)

Abstract: Euphemism is a type of figurative language broadly adopted in social media and daily conversations. People use euphemism for politeness or to conceal what they are discussing. Euphemism detection is a challenging task because of its obscure and figurative nature. Even humans may not agree on if a word expresses euphemism. In this paper, we propose to employ bidirectional encoder representations transformers (BERT), and relational graph attention network in order to model the semantic and syntactic relations between the target words and the input sentence. The best performing method of ours reaches a Macro-F1 score of 84.0 on the euphemism detection dataset of the third workshop on figurative language processing shared task 2022.

## introduction euphemism is a sophisticated language phenomenon in which one usually uses a polite word or expression instead of a more direct one to avoid shocking or upsetting someone 1 . for example, "we are very sorry that he has passed away". here, "pass away" does not mean dissipation intuitively, but death, which can make unpleasant things sound more polite. due to its obscure and figurative nature, euphemism detection which aims to predict a text as euphemism or non-euphemism becomes a particularly challenging classification task. with the usage of euphemisms becoming prevalent on social media and in daily conversation, euphemism detection has received growing research attention to facilitate the understanding of natural language's sentiment and semantics. felt and riloff (2020) make the first attempt to recognize euphemisms and dysphemisms. they identify synonym phrases of given seed euphemism-related phrases by a weakly supervised bootstrapping algorithm and then classify 1 https://www.ldoceonline.com/dictionary/euphemism the phrases using sentiment cues and contextual sentiment analysis. with the advent of pre-trained language models (plms), euphemism detection methods based on plms such as bert @xcite have been proposed. @xcite propose an automatic euphemistic phrase detection method without human effort. they first extract quality phrases and select euphemistic phrase candidates by computing embedding similarities. then they use spanbert to rank and classify all candidates. despite existing work have achieved promising results, there are still several challenges to tackle. on the one hand, existing euphemism detection work mainly focus on mining characteristics of target words/phrases that triggered the euphemism phenomenon. they emphasize too much the euphemism of target words while ignoring the context circumstances where the target words sit. on the other hand, the first step of these methods is often to extract euphemism candidate words or phrases based on domain expertise or existing data annotations. if the first step is not done well, it will influence the subsequent classification and ranking, which may cause error propagation and lead to poor performance. we observe that euphemisms are essentially polysemy. in this sense, we argue that the meanings of euphemism target words/phrases are closely related to the context in which they are located semantically and syntactically. shed light on the great performance achieved by bert and graph neural network @xcite on the aspect-based sentiment analysis task, we propose to employ bert and relational graph attention network (rgat) @xcite to deal with euphemism detection. specifically, our model contains two isolated sub-models, bert-concat and rgat-bert. for bert-concat, the model's input is the concatenation of the input sentence and target words. we use bert-concat to enhance the information of target words and capture the sequential semantic knowledge of the input sentence and target words. rgat-bert is adopted mainly to capture the syntactic information between target words and their corresponding contexts. the graph is built on the dependency tree. to enhance the syntactic connections between target words and the essential contexts, rgat reshapes the dependency tree in which target words are root. it also prunes the reshaped tree to avoid the noise that unimportant contexts bring. finally, we design a voting mechanism to ensemble the results of the two sub-models, which can leverage the advantages of the two. we conduct experiments on the euphemism detection dataset. empirical experimental results demonstrate the effectiveness of our proposed method. we ended up fourth in the third workshop on figurative language processing shared task 2022. ## related work in this section, we briefly review the related work on euphemism detection. existing work mainly focus on identifying euphemistic words. @xcite provide an unsupervised word embedding's similarity method to identify euphemisms (code words) in hate speech. felt and riloff (2020) use sentiment analysis to recognize the euphemistic and dysphemistic language. they adopt a bootstrapping algorithm for finding near-synonym phrases and then classify the collected phrases as euphemistic, dysphemistic, or neutral using lexical sentiment cues and contextual sentiment analysis. with the advent of pre-trained language models, a lot of euphemism detection methods based on plms have been proposed. @xcite propose a self-supervised euphemistic detection method. they first extract candidate phrases from a base corpus and then filter out ones associated with euphemistic seed phrases through embedding similarity computing. finally, they use pre-trained language models to classify these phrases. similar to @xcite , @xcite rely on a set of predefined seed phrases, which may not be generalized to different datasets. @xcite improve @xcite 's approach by adding an automatic paraphraser. @xcite investigate gender differences in euphemism usage and they find that women do not use euphemisms more than men through empirical analysis. @xcite present a corpus of potentially euphemistic terms, which promotes the development of euphemism detection. we observe that most work on euphemism detection focus on euphemistic terms. they pay less attention to the contexts and the connections between euphemistic terms and their corresponding contexts in a sentence, which may lose important information. ## model in this section, we introduce our method for euphemism detection in detail. the overview of our proposed method is shown in figure 1 . we first introduce the pre-processing of the dataset, and then the bert model and the rgat-bert model. finally we elucidate the model ensembling process. ## experiment in this section, we will introduce the dataset and experimental settings, and then analyze the results. ## conclusion in this paper, we have proposed to leverage transformers and relational graph attention networks to detect euphemisms. specifically, on the one hand, we utilize bert-concat to capture sequential semantic information between target words and their corresponding contexts. on the other hand, we adopt rgat-bert to learn the syntactic connections between target words and essential contexts. experimental results show that ensembling the two sub-models can achieve promising performance on the euphemism detection shared task of the third workshop on figurative language processing.