Exploring Euphemism Detection in Few-Shot and Zero-Shot Settings (2022)

Abstract: This work builds upon the Euphemism Detection Shared Task proposed in the EMNLP 2022 FigLang Workshop, and extends it to few-shot and zero-shot settings. We demonstrate a few-shot and zero-shot formulation using the dataset from the shared task, and we conduct experiments in these settings using RoBERTa and GPT-3. Our results show that language models are able to classify euphemistic terms relatively well even on new terms unseen during training, indicating that it is able to capture higher-level concepts related to euphemisms.

## introduction euphemisms are figures of speech which aim to soften the blow of certain words which may be too direct or too harsh @xcite . in the emnlp 2022 figlang workshop euphemism shared task, participating teams are given a set of sentences with potentially euphemistic terms (pets) enclosed in brackets, and the task is to classify whether or not the pet in a given sentence is used euphemistically. in this task/dataset, however, there are many pets which are repeated throughout both the training and testing sets (more details in section 3). in addition, several pets are classified as euphemistic almost 100% of the time during training. this raises an important question: is the model actually learning to classify what a euphemism is, or is it simply reflecting back things it has seen repeatedly during training? how do we know if the model we train can truly capture the essence of what a euphemism is? even among humans, this is a very nontrivial task. if one hears the phrase "lose one's lunch" for the first time, for example, it may not be immediately obvious that it is a euphemism for throwing up. however, when used in a sentence, the context clues together with an understanding of the meanings of the words "lose" and "lunch" will allow a human to piece together the meaning. for a machine to be able to do this, however, is not trivial at all. to this end, we test this by checking whether a model can correctly classify pets it has never seen during training. this leads us to our fewshot/zero-shot setting. the two key contributions of our paper are as follows: 1) we propose and formulate the few-shot and zero-shot euphemism detection settings; and 2) we run initial baselines on these euphemisms using roberta and gpt-3, and we present a thorough analysis of our results. ## related work compared to other figures of speech like similes @xcite and metaphors @xcite , work on euphemisms has been limited. recently, @xcite ; @xcite released a new dataset of diverse euphemisms and conducted analysis on automatically identifying potentially euphemistic terms. in the past, felt and riloff (2020) used sentiment analysis techniques to recognize euphemistic and dysphemistic phrases. other studies also focused on specific euphemistic categories such as hate speech @xcite and drugs @xcite . in terms of zero-shot figurative language detection, the existing literature has also been quite limited. the few existing studies @xcite mostly focus on metaphors and on lowresource settings. this leaves out less common figures of speech such as euphemisms, and the lowresource formulation is also not exactly identical to the zero-shot setting we explore in this paper. ## task and dataset our task is similar to the figlang 2022 workshop shared task on euphemism detection. given a sentence containing a potentially euphemistic term (pet), we want to determine whether the pet is used euphemistically. the key difference with our task is that we perform the binary classification on a few-shot/zero-shot setting. similarly, we use the dataset proposed by @xcite , which contains 1965 sentences with pets, split across 129 unique pets and 7 different euphemistic categories (e.g. death, employment, etc.) furthermore, the dataset also contains additional information such as the category and the status of the pet ("always euph" vs "sometimes euph"). ## results and analysis observe that the macro precision is 0.5 for all 3 models, which tends to happen when the distribution is very skewed and gets a precision of exactly 1.0 for one class and exactly 0.0 for the other class. meanwhile, for the "substances" category performing well, we speculate that this could be because a lot of these words are quite common. words like "weed" and "sober" are used quite commonly, as opposed to other euphemisms, which are less commonly used in everyday conversations (e.g. "ethnic cleansing" is a rare phrase). ## conclusion and future work in this paper, we explored zero-shot and fewshot settings for the euphemism detection task. we formulated the problem settings and crafted zero-shot and few-shot datasets from the emnlp 2022 figlang workshop euphemism shared task dataset. we tried two type of models, namely roberta and gpt-3. we found promising results that these language models (especially fine-tuned roberta) were able to perform quite well, even on completely unseen euphemistic terms. while our results were overall good, the results for gpt-3 were quite poor. in the future, we believe that further prompt engineering or prompt tuning will definitely be helpful in improving the performance of @xcite . furthermore, this idea of few-shot and zeroshot detection is not exclusive to euphemisms. we believe that checking the performance of language models to classify unseen examples is something that will be important to check for a lot of figures of speech and will be important in our quest to process and generate figurative text.