Adversarial Perturbations Augmented Language Models for Euphemism Identification (2022)

Abstract: Euphemisms are mild words or expressions used instead of harsh or direct words while talking to someone to avoid discussing something unpleasant, embarrassing, or offensive. However, they are often ambiguous, thus making it a challenging task. The Third Workshop on Figurative Language Processing, colocated with EMNLP 2022 organized a shared task on Euphemism Detection to better understand euphemisms. We have used the adversarial augmentation technique to construct new data. This augmented data was then trained using two language models: BERT and longformer. To further enhance the overall performance, various combinations of the results obtained using longformer and BERT were passed through a voting ensembler. We achieved an F1 score of 71.5 using the combination of two adversarial longformers, two adversarial BERT, and one non-adversarial BERT.

## introduction euphemisms are mild words or expressions used instead of harsh or direct words while talking to someone to avoid discussing something unpleasant, embarrassing, or offensive. they are often used as a sign of politeness while discussing sensitive or taboo topics @xcite , for instance, using the term "let go" instead of the word "fired," using "put down" instead of "euthanized," or any similar phrase that would make it sound less unappealing or unpleasant @xcite . euphemism can also be employed to disguise the truth @xcite to minimize a threatening situation to create a favorable image. for instance, when the phrase "enhanced interrogation techniques" is used, they mean "torture" or use "armed conflict" instead of "war". this figurative behavior of euphemisms makes it ambiguous and challenging for natural language processing techniques to handle these words since they can be interpreted literally in some situations. moreover, humans might disagree with what constitutes a euphemism @xcite . in the past, many computational approaches for processing have been employed. a sentiment analysis-based approach was used by @xcite to handle x-phemisms (a term used to refer to both euphemisms and dysphemisms). in their work, they found synonym pairs and used a weakly supervised bootstrapping algorithm to generate semantic lexicon. these lexicons were then used to classify phrases as euphemistic, dysphemistic, or neutral. @xcite worked on detecting euphemisms used for dug names on the internet and identifying the terms these euphemisms refer to. similarly, @xcite also worked on a similar problem statement. however, @xcite and @xcite interpreted euphemisms as code words, which is different from those of the shared task organizers. both @xcite and @xcite considered the detection and identification of euphemism as a masked language model (mlm) problem where they filtered out words that did not fit their list of euphemisms. this paper defines our participation in the euphemism detection shared task @xcite organized for the third workshop on figurative language processing co-located with emnlp 2022. we have used the adversarial augmentation technique in combination with transformers to detect euphemisms. a detailed explanation of our proposed approach is given in section 3. ## task & data description euphemism detection is a binary classification shared task that focuses on detecting euphemisms in a given input statement. this shared task aims to study the performance of natural language models on euphemisms. the data provided in the shared task comprised a euphemism corpus created by preprocessing attack module adversarial perturbation user module augmentation longformer tl: 600 bert tl: 450 longformer tl: 600 bert tl: 450 bert tl: 500 ensembler not euphemism euphemism input non-augmented data non-augmented data ( @xcite . the raw data used for the creation of this dataset is extracted from the corpus of global web-based english (glowbe) @xcite , which contains text data from websites, blogs, and forums of twenty different english-speaking countries. the training data provided to the participants comprised 1572 sentences, of which 466 were labeled "0," and the rest were labeled "1". the testing data consists of 393 data points. the participants were expected to classify the sentences into "0" (not euphemistic) and "1" (is euphemistic) classes. ## methodology this section gives a detailed description of the pipeline proposed. section 3.1 and section 3.2 provides a detailed overview of the preprocessing performed on the data and the augmentation technique used before passing the data through the models. section 3.4 details the models used for the training. ## results and discussion euphemism in speech is generally difficult to identify semantically for human beings and thus makes it even more challenging task for the ai to map the understanding and undergo right identification. our submission aims to handle the task of identifying the euphemism in a given sentence by modelling longformer and bert in an adversarial setup. ## conclusion in this paper, we proposed an adversarial perturbated (textattack) bert and longformer model, which aims to create a robust model capable of identifying the euphemism in text. we experimented with different token lengths and eventually created a voting ensemble model that combined our other experiments into a single encapsulation. the ensemble of two adversarial longformers, two adversarial bert, and one non-adversarial produced an f1 score of 0.715, which was our best submission. the use of dual context input to the models falls short of the expected performance boost and motivates us to look further into the concept of using multiple representations. we aim to experiment with different methods to combine these representations into a single exemplary representation that can pass into these language models to solve the downstream tasks.