Figuratively Speaking: Authorship Attribution via Multi-Task Figurative Language Modeling (2024)

Abstract: The identification of Figurative Language (FL) features in text is crucial for various Natural Language Processing (NLP) tasks, where understanding of the author’s intended meaning and its nuances is key for successful communication. At the same time, the use of a specific blend of various FL forms most accurately reflects a writer’s style, rather than the use of any single construct, such as just metaphors or irony. Thus, we postulate that FL features could play an important role in Authorship Attribution (AA) tasks. We believe that our is the first computational study of AA based on FL use. Accordingly, we propose a Multi-task Figurative Language Model (MFLM) that learns to detect multiple FL features in text at once. We demonstrate, through detailed evaluation across multiple test sets, that the our model tends to perform equally or outperform specialized binary models in FL detection. Subsequently, we evaluate the predictive capability of joint FL features towards the AA task on three datasets, observing improved AA performance through the integration of MFLM embeddings.

## introduction figurative language (fl) constructs, such as metaphor, simile, and irony, are common in various forms of communication, such as literature, poetry, and speech. their use can enrich the meaning, creativity, and persuasiveness of a message and help to achieve an intended impact on the reader. the use of certain forms of fl in writing reflects the authors' style and background, including their education, personality, social context, and worldviews. therefore, we hypothesise that the choice of figurative language features in (written) communication may reveal the writer's cognitive and linguistic basis that underlie their production, and how their selection is influenced by the context, the intention, and the emotion of the writer. in this paper, we introduce a multi-task classifi-cation model designed to detect multiple figurative language (fl) features in a body of text. the first research question (rq1) we seek to answer is: "is a model that is trained to detect multiple fl features simultaneously more effective than multiple specialized models, each trained to detect a specific fl feature?" through our research, we demonstrate that this multi-task model is indeed more effective than using several binary models. in our research, we utilize 13 publicly available datasets to train and evaluate both binary and multitask models. we deliberately opted against integrating additional datasets specifically designed for metaphor detection, which is only one of the phenomena we study. the rationale behind this decision was creating a more balanced training data, which otherwise would have been disproportionately skewed our study towards metaphor detection, given the substantially more resources dedicated to this phenomenon. at the same time, the lack of annotated corpora for other figurative language features such as personification, metonymy, oxymoron, etc. necessarily limited our initial study to the six fl constructs that are generally well represented among these 13 datasets: metaphor, simile, idiom, sarcasm, hyperbole, and irony. all our binary models and the multi-task model are based on roberta @xcite . after training the specialized binary models on the combined datasets, we used them to automatically label our training corpora with all applicable fl features. this multi-label dataset was then used to train our multi-task model. afterwards, we compare our multi-task figurative language model (mflm) against the binary classifiers on the 13 test sets. the results showed that mflm matched or outperformed the binary classifiers in five test sets and achieved higher task-specific performance than the binary models in another three test sets, which suggests that these features are not independent from one another. after training our multi-task figurative language classifier, we put forward a second research question (rq2): "does the incorporation of figurative language (fl) features enhance performance in authorship attribution (aa) tasks?" to answer this, we evaluate the impact of the fl features learned by our multi-task figurative language model (mflm) on three publicly available aa datasets, each consisting of documents with varying topical content and number of authors. for each dataset, we train multi-layer perceptron (mlp) classifiers, using mflm sentence embeddings and other baselines as input features. the baselines consist of classical stylometric features, character and word n-gram tf-idf vectors, and generic sentence embeddings. our results demonstrate that the aa task performance is indeed improved by combining mflm embeddings with other baselines. to our knowledge, this work is the first to examine the applicability of fl features in aa. we should note here that we did not expect that the fl features alone would be sufficient to perform aa; rather we set off to demonstrate that incorporating combined fl features improves aa performance when integrated with more basic stylistic features, particularity for longer texts. the results show that the latter is generally true; however, we found that the fl features perform nearly as strong and sometimes better on their own. this supports our initial stipulation that fl use is highly personalized, and thus an excellent predictor of authorship. we make our code and data available in our github repository foot_0 . ## related work most of the previous studies on figurative language (fl) feature detection focus on the features independently. an earlier work, @xcite , used lexical semantic features of the words to discriminate metaphors from literals. more recently, @xcite utilized metaphor identification theories using roberta to predict whether a word in a sentence is metaphorical or not. a similar shift from linguistic feature based approach to pre-trained language model (plm) based approach is observed in simile detection. @xcite extracted features such as topic-vehicle similarity and imageability to separate similes from literal comparisons. @xcite used bert @xcite and roberta in simile property probing tasks and concluded that the plms still underperformed humans. plms are also applied to the detection of sarcasm @xcite , hyperbole @xcite , irony @xcite , and idiom @xcite . among the studies that work on more than one features, @xcite used datasets crosslabeled with metaphor and hyperbole, and found that the multi-task learning approach performed better than the single-task approach on both features. @xcite rendered the fl detection into a multi-task natural language inference (nli) problem, developed a nli dataset of four fl features, and tested with several experimental systems. @xcite collected datasets on idiom and simile and developed knowledge enhanced roberta-based models. however, their task was to predict the correct continuation of the given narrative, not fl feature detection. @xcite built a dataset covering 9 fl features plus literals. they tested three baseline systems in a multi-class classification task and bert outperformed the other two systems. there is a rich literature in the field of authorship attribution (aa). various methods have been applied to the task, ranging from svm based approaches, such as @xcite , to transformer based models, like @xcite . in pan-2019 cross-domain aa challenge @xcite , most of the submissions used n-gram features (char, word, part-of-speech) and an ensemble of classifiers @xcite . @xcite fine-tuned a bert model for aa task and tested the model on three datasets including imdb-62 @xcite . in a recent review article @xcite , feature based methods and embedding based methods were tested and compared on the same datasets. they used ngrams, summary statistics and co-occurance graphs as features, as well as static char/word embeddings and transformer-based sentence embeddings. ## figurative language modeling in our study, we investigate the potential benefits of combining figurative language (fl) features as opposed to analyzing each feature independently. to answer our first research question, we examine whether training a fl classification model capable of jointly labeling text with relevant features would outperform a singular binary model specialized in detecting only one feature. this idea stems from noticing that in both spoken and written language, individuals intertwine various elements of figurative speech to effectively convey their intended message. consequently, fl features frequently co-occur, and understanding the interplay between these features may offer valuable insights for improving their identification accuracy. this research builds upon prior studies that explored the simultaneous detection of metaphors and sarcasm, as well as hyperbole and sarcasm. in our investigation, we aim to simultaneously learn to detect six distinct fl features: metaphors, simile, sarcasm, hyperbole, idiom, and irony. ## authorship attribution we proceed to investigate the effectiveness of our multi-task figurative language model (mflm) in the closed-case authorship attribution (aa) downstream task. aa involves classifying texts to determine their respective authors from a known set of candidates. specifically, given a training corpus consisting of n authors, the objective is to predict the author of each document in the test set by selecting from the set of n authors. our second research question proposes that embeddings incorporating figurative language features will enhance performance in the aa task. this concept extends from stylometric analysis @xcite , which traditionally concentrates on discerning patterns within written text. stylometric analysis examines various aspects of writing style, including word selection, sentence construction, punctuation usage, and vocabulary preferences. to the best of our knowledge, our study is the first of its kind to utilize a transformer model that has been fine-tuned for multi-task fl classification, towards the aa task. previous research in this area minimally explored the applicability of fl features for this specific task. ## conclusion this study investigated two research questions regarding the detection and application of figurative language (fl) features in machine learning. firstly, we explored whether a multi-task model trained to simultaneously detect multiple fl features (metaphor, simile, idiom, sarcasm, hyperbole, and irony) could outperform individual models specialized for each feature. by leveraging roberta-large and a multi-label training dataset derived from binary classifiers, our multi-task figurative language model (mflm) achieved superior performance on 8 out of 13 test sets, particularly excelling in detecting simile, idiom, irony, and hyperbole. this finding highlights the increased effectiveness of a unified approach for comprehensive fl detection. secondly, we examined the potential of incorporating fl features to enhance performance in authorship attribution (aa) tasks. utilizing three diverse aa datasets and multi-layer perceptron (mlp) classifiers, we evaluated the contribution of mflm sentence embeddings alongside various baseline features like stylometric features, sbert embdeddings and word and character ngram vectors. the results showed the competitive performance achieved by mflm embeddings alone, while their combination with other features yielded consistent performance improvements across nearly all cases. this strongly supports the second research question, indicating the positive impact of integrating fl features in aa tasks. our study offers valuable insights into the effectiveness of multi-task learning for comprehensive fl detection and the potential of fl features to improve aa tasks. further research could explore the applicability of mflm to additional nlp tasks, such as sentiment analysis and information retrieval. moreover, future studies could investigate the impact of incorporating additional fl features into a single classification model, such as personification, metonymy, onnomatopoeia, etc., as well as domain-specific knowledge for even more refined fl detection and application. in the process of constructing annotated corpora for training machine learning algorithms for automatic figurative language detection, it's crucial to consider the interpretive discrepancies between experts and non-experts. the annotations found in the collection of datasets used in this study are all taken as ground truth of equal importance, potentially leading towards a biased fl detection model. an expert, with their nuanced understanding, can identify subtle metaphors and idioms that may elude an ordinary reader. however, non-experts, influenced by their unique cultural backgrounds and personal experiences, may interpret figurative language differently @xcite . for instance, certain phrases may have specific connotations in one culture but be meaningless in another. similarly, a person's familiarity with a subject matter can greatly influence their understanding of related figurative language. therefore, to ensure a more accurate and comprehensive analysis of figurative language, these factors must be taken into account. future work will address this issue by conducting qualitative and quantitative analyses on the annotated datasets. in our methodology, we employ specialized binary models for each feature, trained on our combined datasets, to predict figurative language labels for the training examples used in our multi-task model. this approach, while effective, can lead to error propagation, resulting in incorrect predictions from our model. however, our evaluation and manual error analysis indicate that our multi-task model's predictions are often reasonable, with errors frequently attributable to incomplete human annotations. the second part of our study applies the embeddings from our multi-task fl model to the aa task. we train mlp classifiers using document vectors as features on three publicly available datasets, each focusing on a different topic: movie reviews, corporate/industrial topics, and fan fiction. however, these topics are not very diverse, which could introduce bias into the datasets with respect to authorship. for instance, in the fan fiction dataset, some authors may exclusively write "harry pot-ter" fan fiction, which could skew the evaluation of different features. lastly, it is important to note that the predictions of deep neural language models, such as the ones used in our study, are often difficult to interpret and explain. this lack of interpretability is a common challenge in the field and is another limitation to consider in our work. cynthia van hee, els lefever, and véronique hoste. 2018. semeval-2018 task 3: irony detection in english tweets. in proceedings of the 12th international workshop on semantic evaluation, pages 39-50. byron c wallace, laura kertz, eugene charniak, et al. 2014. humans require context to infer ironic intent (so computers probably do, too). in proceedings of the 52nd annual meeting of the association for computational linguistics (volume 2: short papers), pages 512-516. mengfei yuan, zhou mengyuan, lianxin jiang, yang mo, and xiaofeng shi. 2022. stce at semeval-2022 task 6: sarcasm detection in english tweets. in proceedings of the 16th international workshop on semantic evaluation (semeval-2022), pages 820-826, seattle, united states. association for computational linguistics. c udny yule. 2014. the statistical study of literary vocabulary. cambridge university press. yunxiang zhang and xiaojun wan. 2021. mover: mask, over-generate and rank for hyperbole generation. arxiv preprint arxiv:2109.07726.