Transfer-Learning German Metaphors Inspired by Second Language Acquisition (2025)

Abstract: A major part of figurative meaning prediction is based on English-language training corpora. One strategy to apply techniques to languages other than English lies in applying transfer learning techniques to correct this imbalance. However, in previous studies we learned that the bilingual representations of current transformer models are incapable of encoding the deep semantic knowledge necessary for a transfer learning step, especially for metaphor prediction. Hence, inspired by second language acquisition, we attempt to improve German metaphor prediction in transfer learning by modifying the context windows of our input samples to align with lower readability indices achieving up to 13% higher F1 score.

## introduction figurative language detection is one of the most crucial tasks in the current digital conversational landscape. however, computationally, it remains also one of the most challenging tasks. comprehensive resources to train computational models for figurative language detection are generally rare. further, most existing work is performed on english language textual data. some works investigate metaphor recognition in languages other than english @xcite . we focus on applying and testing transfer learning techniques to continuously correct for this imbalance in figurative language prediction. we think that, due to the conceptual nature of metaphors @xcite , it is possible to transfer metaphoric meaning given a sufficient amount of data that is capable of encoding this conceptual nature. the study in this paper is designed as follows: first, we address the motivations of this research by presenting the readability indices of the predicted test samples of a prior study @xcite . then, we modify the test samples according to these insights by trimming the observed contexts. this means, shortening the input. last, we re-apply the multi-lingually pre-trained transformer models to determine how the sample modification affects the performance of the multilingual classifiers. ## methodology we already learned that computational approaches work well in semantically "coarse-grained" tasks such as semantic similarity prediction @xcite @xcite or authorship attribution @xcite , because they are well capable to distinguish the meaning of a word in different contexts. in figurative language identification, contextual representation is also a good input for a classifier to predict whether a word is meant figuratively or literally @xcite @xcite . transfer learning typically makes use of a wellresourced source language to train a classifier on, afterwards, the trained model is applied to predict metaphors in a low/less-resourced language. however, there are two major problems to make figurative language prediction work cross-lingually: first, only a few larger (lexicondependent) annotated datasets for training in the source language are available foot_0 ; second, the translation models of today's transformers are incapable to encode the deep semantic knowledge required for transfer identification of figurative language @xcite . as syntax is the structural representation of meaning, one can carefully state that sentences of more complex syntax usually also entail more complex semantics. as such, "adding" tokens to a string also often (not always) means to "add" semantics to the meaning of a phrase or sentence. this can be partially validated by @xcite who investigate the role of compositionality and lexical semantics in determining informativeness at the phrasal level. as we understand that the transformer models may need to be presented with "easier" (shorter, less complex) samples because this is the case when learning a new language, we attempt to improve german metaphor prediction in transfer learning by modifying the context windows of our input samples to align with lower readability indices. in particular, trimming the context of a potential metaphoric expression can aid in the model's focus on nearby domain-related context while long-distance context may be less relevant, and the preserving of the sentence's global meaning possibly plays a subordinated role. to backup the latter assumption, we also test a more sophisticated technique by applying @xcite 's gpt2.0-based text simplifier-to the best of our knowledge the only model, applicable for german language text-to our test data. ## results and discussion tab. 2 shows that the best performance increase can be reached with trimming the contextual span for the input representation to 11 tokens. also, a context of 7 enables the models to drastically increase on performance while allowing a window of 10 to each side still results in an increase of up to 6% in f1 (c.f., upper part of tab. 2). the neural simplification (that also preserves a sentence's meaning) achieves up to 9% increase in f1 returning the second highest f1 score. looking at samples from the sbert output, we find that limiting the context can help the model to better focus on local meaning. for example in the form of not labeling words as figurative that actually are not used figuratively. while the following for comparison, in the lower part of tab. 2, we also list the results of fine-tuning in german, based on the 908 de language samples, which we split into train, validation and test sets @xcite . we can see that fine-tuning on target data and language after training in the vua data brings the best results @xcite . when we test whether fine-tuning on target domain english language data improves the test results in our german language data, we find a positive effect. especially, xlm-roberta shows the ability to well generalize to language-independent data points when the source (training) and target (testing) domain remain the same. this can be explained by the dynamic masking process during roberta's initial training process. however, in semantically challenging set-ups, this flexibly rather prevents roberta from retrieving unknown items, as can be seen in the results of applying roberta to our german metaphor data after only training on the vua corpus (second line of tab. 2 ). shortening is not simplification: it is not always the case that a metaphor is difficult to extract because a sentence is syntactically complex, nor is it always true that humans understand shorter sentences better than longer ones. but, sentence simplification usually divides up complex content into many shorter sentences and this also improves metaphor recognition for a computational model. further, our trimming approach is technically simple and streamlined and shows already good results. we further will elaborate on a quantitative approach that incorporates advanced syntax-tree rules into our window-trimming technique. ## conclusion we demonstrated a computationally simple approach to correct input representation to make them shorter, hence, easier for the model to understand, because-as in second language acquisition, we learned that the translation representations of transformer models have some difficulty in "understanding" the deep semantics required for figurative meaning classification. we also applying a gpt-based simplifier. we achieve an increase of up to 13% (11-token context) and up to 9% (neural simpl.) in f1 and find that the sentence transformer models perform best in metaphor prediction. in future, we plan to apply didacticallyinformed approaches that utilize linguistic, comparative, and didactic knowledge while being applicable to quantitative methods as well.