A Dataset for Metaphor Detection in Early Medieval H ebrew Poetry (2024)

Abstract: There is a large volume of late antique and medieval Hebrew texts. They represent a crucial linguistic and cultural bridge between Biblical and modern Hebrew. Poetry is prominent in these texts and one of its main characteristics is the frequent use of metaphor. Distinguishing figurative and literal language use is a major task for scholars of the Humanities, especially in the fields of literature, linguistics, and hermeneutics. This paper presents a new, challenging dataset of late antique and medieval Hebrew poetry with expert annotations of metaphor, as well as some baseline results, which we hope will facilitate further research in this area.

## introduction the hebrew language has a long and rich history, from biblical hebrew, through rabbinic-medieval hebrew, to modern hebrew. in this work, we present a corpus consisting of hebrew liturgical poetry from the fifth to eighth centuries ce, also known as piyyut (from greek poietes, to create, versify; plural: piyyutim). the piyyutim in the corpus were reconstructed throughout most of the twentieth century by various scholars from manuscripts of the cairo genizah, a medieval repository of jewish texts @xcite . since poetry was a prominent genre in late antique and medieval hebrew literature, the corpus is rich in figures of speech like similes and metaphors. active research in this area is conducted by scholars in the humanities, especially digital humanities, who wish to understand not only the literal meaning of a text but also its figurative meaning @xcite . at present, texts are annotated manually, a time-consuming and laborintensive process. scholars of hebrew literature and hebrew linguists would thus benefit greatly from a tool that automatically detects figurative language in these texts. furthermore, such tools could be used by non-specialists who want to better understand these texts by highlighting figurative language. since the literary and linguistic tradition of piyyut runs throughout the middle ages, working on the early strata of this tradition would enable us to extend the impact of metaphor detection also to later periods and other genres. yet, to the best of our knowledge, there are no previous studies that deal with this task, in either modern or pre-modern hebrew. to fill this gap, the main contribution of this work is a medieval hebrew dataset of hebrew liturgical poetry with metaphor annotations. the dataset consists of two units of ancient piyyut, with 309 poems and 73,179 words, with expert annotations for metaphorical expressions. despite its relatively small size, the corpus contains 15% of the digitized piyyutim and is the only metaphorannotated corpus available in hebrew. we develop and evaluate several transformerbased models for detecting metaphors in the dataset, based on two pre-trained hebrew language models: alephbert, which was pre-trained on modern hebrew @xcite , and berel, pre-trained on ancient jewish texts that are closer in style to the piyyut texts @xcite . we substantially improve naïve baselines, with our best model achieving f1 scores of 48.7 and 49.4 on the two corpora. considering the difficulty of the task, attested through an inter-annotator agreement study we conducted, we find the results encouraging while leaving ample room for improvements. ## conclusion we presented a corpus of medieval hebrew poetry with metaphor annotations. the corpus can serve literary scholars who wish to study figurative language use in this genre. we also evaluated basic approaches for automatic metaphor detection based on this corpus, emphasizing the importance of adapting models to this particular genre. models such as these have some practical applications. by automatically detecting metaphors in piyut texts, people can better understand these ancient texts. furthermore, these tools may allow experts to semi-automatically annotate more texts. we hope to facilitate further research in this area, both in designing more sophisticated methods for metaphor detection in this challenging corpus and in improving the workflow of literary scholars interested in this body of texts. figures 1 and 2 show histograms of texts in the two corpora, binned by the ratio of metaphor words they contain. while a few texts contain a very high ratio of metaphor words, most texts have a small ratio.