TEDB System Description to a Shared Task on Euphemism Detection 2022 (2022)

Abstract: In this report, we describe our Transformers for euphemism detection baseline (TEDB) submissions to a shared task on euphemism detection 2022. We cast the task of predicting euphemism as text classification. We considered Transformer-based models which are the current state-of-the-art methods for text classification. We explored different training schemes, pretrained models, and model architectures. Our best result of 0.816 F1-score (0.818 precision and 0.814 recall) consists of a euphemism-detection-finetuned TweetEval/TimeLMs-pretrained RoBERTa model as a feature extractor frontend with a KimCNN classifier backend trained end-to-end using a cosine annealing scheduler. We observed pretrained models on sentiment analysis and offensiveness detection to correlate with more F1-score while pretraining on other tasks, such as sarcasm detection, produces less F1-scores. Also, putting more word vector channels does not improve the performance in our experiments.

## introduction a shared task on euphemism detection @xcite is the first installment of a natural language processing (nlp) shared task on a particular figurative language detection, euphemism. figurative languages, including metaphors, synecdoches, idioms, puns, hyperbole, similes, onomatopoeia, and others, are word uses where the meaning deviates from the literal meaning to convey a complicated, creative and evocative message without directly stating it. in addition, figurative language might use contexts such as relations to other things, actions, social experiences, or images. figurative languages are ubiquitous since they are filled in countless of our everyday activities without notice @xcite . euphemisms are mild or indirect words or phrases being used in place of offensive or unpleas- ## experimental setup our input consists of a three-sentence utterance, the sentence before, after, and the sentence containing the euphemistic term. we did not observe any improvements from removing any special characters including the '<' and '>' symbols around the euphemistic term given in the dataset. we used the maximum input length of 150 tokens since we found that it is the number that fits well as our heuristics with the gpu memory for many reasonable batch sizes (4-20 in our cases). also, it seems to cover most data instances given the histogram plotting in figure 1 . we sampled the model at the end of each epoch. the dataset has 1572 training instances and 393 test instances. all of our experiments were done in the google colab setting on nvidia tesla t4 gpus. we used the batch size in the range of 4 -20 and the learning rate for an adamw optimizer @xcite in the set of {2.5e -5, 2e -5, 1e -5, 7.5e -6} for all experiments. we considered linear annealing scheduler and cosine annealing scheduler with restart. the cycle number is in the set of {5, 8}. also, adding a warm-up step does not make any difference so we set the warm-up step to zero in all experiments. ## conclusion this report describes our baseline systems for a shared task on figurative language processing 2022, euphemism detection. our best result is from a single-channel kimcnn model using 'cardiffnlp/twitter-roberta-base-sentimentlatest', pretrained again for euphemism detection, as a feature extractor. we observed more stability and faster convergence from this training schema. our results on pretrained transformer models are likely to confirm the previous works @xcite @xcite that euphemism relates with sentiment and offensiveness. still, we also observed that finetuning a sentiment-based pretrained language model, which pretrained with a rather small dataset, does not perform well.