MED s for PET s: Multilingual Euphemism Disambiguation for Potentially Euphemistic Terms (2024)

Abstract: Euphemisms are found across the world’s languages, making them a universal linguistic phenomenon. As such, euphemistic data may have useful properties for computational tasks across languages. In this study, we explore this premise by training a multilingual transformer model (XLM-RoBERTa) to disambiguate potentially euphemistic terms (PETs) in multilingual and cross-lingual settings. In line with current trends, we demonstrate that zero-shot learning across languages takes place. We also show cases where multilingual models perform better on the task compared to monolingual models by a statistically significant margin, indicating that multilingual data presents additional opportunities for models to learn about cross-lingual, computational properties of euphemisms. In a follow-up analysis, we focus on universal euphemistic “categories” such as death and bodily functions among others. We test to see whether cross-lingual data of the same domain is more important than within-language data of other domains to further understand the nature of the cross-lingual transfer.

## introduction euphemisms are a linguistic device used to soften or neutralize language that may otherwise be harsh or awkward to state directly (e.g. "between jobs" instead of "unemployed", "late" instead of "dead", "collateral damage" instead of "war-related civilian deaths"). by acting as alternative words or phrases, euphemisms are used daily to maintain politeness, mitigate discomfort, or conceal the truth. while they are culturally-dependent, the need to discuss sensitive topics in a non-offensive way is universal, suggesting similarities in the way euphemisms are used across languages and cultures. this study explores whether multilingual models take advantage of such similarities when processing euphemisms. we use the multilingual transformer model xlm-roberta-base @xcite , or "xlm-r", as our deep learning model, and work with four languages (mandarin chinese, american english, spanish, and yorùbá) that encompass a diverse range of linguistic and cultural backgrounds. in our experiments, we focus on the euphemism disambiguation task, in which potentially euphemistic terms (pets) are classified as euphemistic (1) or not (0) in a given context (e.g., "let go" may mean "fired" in some contexts, but not all in other contexts). models are trained on labeled data from a single, or multiple languages, and evaluated separately on all four languages. our contributions are as follows: (1) we augment existing chinese and spanish datasets started by @xcite and perform additional analyses (section 3). (2) we run classification experiments and find cases of cross-lingual transfer (i.e. a model trained on one language can classify instances in another language), as well as an overall performance improvement when training models on multiple languages versus one (section 4). (3) we perform a follow-up experiment in which we find signs that the cross-lingual transfer may be related to euphemistic category (section 5). these results suggest that xlm-r picks up on "knowledge" about euphemisms which it can not only transfer, but also synergize across languages. ## related work in recent years, there has been growing interest in computational approaches to euphemism detection in the natural language processing (nlp) community. felt and riloff (2020) introduced the recognition of euphemisms and dysphemisms using nlp, generating near-synonym phrases for sensitive topics. @xcite proposed euphemism detection and identification tasks using masked language modeling with bert. @xcite created a corpus of potentially euphemistic terms (pets). @xcite developed a linguistically driven approach for identifying pets using distributional similarities. bert-based systems that participated in a shared task on euphemism disambiguation showed promise @xcite . keh (2022) experimented with classifying pets unseen during training. @xcite perform transformer-based euphemism disambiguation experiments, exploring vagueness as one of the properties of euphemisms. other existing work has explored the multilingual and cross-lingual transfer capabilities of large language models (llms). @xcite found that multilingual llms rely on data from multiple languages to a large extent, learning both complementary and reinforcing information. @xcite found cases where transfer learning from out-of-language data in a particular domain performed better than same-language data in a different domain. ## multilingual corpus of euphemisms for our data, we use the multilingual mandarin chinese (zh), american english (en), spanish (es), and yorùbá (yo) euphemism datasets created by @xcite . in these datasets, text examples containing pets are annotated by native speakers with a 0 or a 1 (i.e. a euphemistic or non-euphemistic usage of the pet). we modify the datasets to become similar to one another in two ways: firstly, yorùbá lacked "boundary tokens" to the left and right side of pets, so we add them in where possible; for some examples (∼25%), the pet tokens were sometimes separated due to yorùbá word order, so multiple pairs of "boundary tokens" were added for these examples. secondly, to balance the number of examples in each language, we augmented the mandarin chinese and spanish datasets. using the guidelines from the original paper, native speakers (who were co-authors) added more pets (40 for chinese and 67 for spanish) and examples (453 for chinese and 900 for spanish) to obtain the final euphemism corpus used for this paper foot_0 . see as can be seen, while the number of examples are fairly balanced across languages, there are still two main differences. one is the number of ambiguous pets; i.e. pets which have both euphemistic and non-euphemistic usages in the dataset. higher numbers of ambiguous pets and examples may contribute to a higher "degree of difficulty" for classification. two, we additionally contribute interrater agreement metrics for the mandarin chinese, spanish, and yorùbá datasets. we recruited 2 native speakers to annotate a random subset of 500 examples from each dataset and then compute krippendorf's alpha (hayes and krippendorff, 2007), α, following the example of @xcite who obtained an alpha of 0.415 for the english dataset. the results can be found in the last column ## experiments with euphemistic category motivated by the question "what is the nature of the cross-lingual knowledge being learned about euphemisms?", we ran a follow-up experiment in which we looked at specific euphemistic categories 3 . we created test sets of examples in which we isolate a single language and a single category, out of a possible 4 categories that had a substantial number of examples in each dataset: physical/mental attributes (attr), bodily functions/parts (body), death (death), and sexual activity (sex). then, we compare two different training settings: (1) training only on same-category, but out-of-language examples ("sc-ool"), and (2) training only on same-language, but out-of-category examples ("sl-ooc"). for all language-category scenarios, there were always fewer sc-ool examples than sl-ooc, so we used the maximum number of sc-ool examples available, down-sampled for the sl-ooc examples, and used a random 90-10 split to create training and validation sets. more detailed metrics regarding the number of examples can be found in appendix c. we use the same parameters as in 4.1, except we increased the early stopping patience to 10 (due to having smaller datasets) and only perform 10 runs for each setting. in ## conclusions and future work in this study, we investigate the multilingual and cross-lingual capabilities of multilingual transformers for euphemism disambiguation. we found cases of zero-shot, cross-lingual learning, and that fine-tuning on multiple languages yields statistically significant improvements for chinese, english, and spanish. this indicates that multilingual approaches may work as a method of data augmentation, which would be particularly useful for data-scarce figurative language tasks (especially for low-resource languages). the results also suggest that some of these patterns are language-specific, and dependent on training settings. more work is needed to test other training parameters (e.g. number of examples) and languages from a variety of families. while it is hard to answer the question "what exactly is being learned about euphemisms crosslingually?", we found preliminary evidence that part of the answer may relate to euphemisms' semantic category. exploring this question further is left to future work, which may be important from both a linguistic and computational perspective.