Sentiment Analysis of Latin Poetry: First Experiments on the Odes of Horace
Authors: Rachele Sprugnoli, Francesco Mambrini, Marco Carlo Passarotti, Giovanni Moretti (Università Cattolica del Sacro Cuore, 2021)
Topic: Sentiment, Emotion, Irony, Hate

Abstract
In this paper we present a set of annotated data and the results of a number of unsupervised experiments for the analysis of sentiment in Latin poetry. More specifically, we describe a small gold standard made of eight poems by Horace, in which each sentence is labeled manually for the sentiment using a four-value classification (positive, negative, neutral and mixed). Then, we report on how this gold standard has been used to evaluate two automatic approaches for sentiment classification: one is lexicon-based and the other adopts a zero-shot transfer approach.

Introduction
The task of automatically classifying a (piece of) text according to the sentiment conveyed by it, known as Sentiment Analysis (SA), is usually performed for purposes such as monitoring contents of social media or evaluating customer experience, by analysing texts like tweets, comments, and micro-blogs.  A still under-investigated yet promising research area where developing and applying SA resources and techniques is the study of literary texts written in historical and, particularly, Classical languages (e.g. Ancient Greek and Latin). Actually, investigating the lexical properties of Classical literary texts is a century-long common practice. However, such investigation can nowadays (1) lead to replicable results, (2) benefit from techniques developed for analysing the sentiment conveyed by any type of text and (3) be performed with freely available lexical and textual resources. As for the latter, the research area dedicated to building and using linguistic resources for Classical languages has seen a substantial growth during the last two decades <cit.>. For what concerns SA, we recently built a polarity lexicon for Latin nouns and adjectives, called LatinAffectus. The current version of the lexicon includes 4,125 Latin lemmas with their corresponding prior polarity value <cit.>. LatinAffectus was developed in the context of the LiLa: Linking Latin project (2018-2023)[<https://lila-erc.eu>] <cit.> which aims at building a Knowledge Base of linguistic resources for Latin based on the Linked Data paradigm, i.e. a collection of several data sets described using the same vocabulary of knowledge description and linked together. LatinAffectus is connected to the Knowledge Base, thus making it interoperable with the other linguistic resources linked so far to LiLa <cit.>.  In this paper we describe the use of LatinAffectus to perform SA of the Odes (Carmina) by Horace (65 - 8 BCE). Written between 35 and 13 BCE, the Odes are a collection of lyric poems in four books. Following the models of Greek lyrical poets like Alcaeus, Sappho, and Pindar, the Odes cover a wide range of topics related to the individual and social life in Rome during the age of Augustus, like love, friendship, religion, morality, patriotism, the uncertainty of life, the cultivation of tranquility and the observance of moderation. In spite of a rather lukewarm initial reception, the Odes quickly became a capital source of influence, in particular as a model of authorial voice and identity.[For an orientation on the vast subject of the fortune and reception of the Odes see Baldo baldohorace2012.]  Considering not only the importance of the Odes in the history of Latin and European literature, but also the diversity of the contents and tones of the poems collected therein, we argue that performing SA on such work can lead to interesting results and might represent a use case to open a discussion about the pros and cons of applying SA techniques and resources to literary texts written in ancient languages.  All data presented in this paper are publicly released: <https://github.com/CIRCSE/Latin_Sentiment_Analysis> .

Conclusion
In this paper we have presented a new GS, made of odes written by Horace, for the annotation of sentiment in Latin poetry. The extension of the manually annotated dataset is one of our future work: the goal is to have a sufficient amount of data to test supervised systems. We have also experimented two different SA approaches that do not require training data: both of them are not able to correctly identify sentences with mixed sentiments, which, in any case, are the most problematic also for human annotators. Table <ref> reports a comparison in terms of precision, recall and F1-score among the lexicon-based approach and the zero-shot classification experiments with both the mBERT and the XML-RoBERTa models. The former performs better on the class whereas the zero-shot method achieves a higher F1-score on the one even if this class is not the most frequent in the training data. Both mBERT and XML-RoBERTa obtain a very high precision on the sentences marked as (0.83 and 1.00 respectively) but the recall is extremely low (0.25 and 0.10 respectively). On the contrary, for the class, the recall is perfect (1.00 for both models) but the precision is very low (0.10 and 0.11 respectively). A manual inspection of the output of the lexicon-based method revealed two main problems of that approach: i) the limited coverage of LatinAffectus and ii) sentiment shifters are not properly taken into consideration. As for the first point, LatinAffectus covers the 43% of nominal and adjectival lemmas in the GS, leaving out lemmas with a clear sentiment orientation. To overcome this issue, we are currently working on the extension of the lexicon with additional 10,000 lemmas. Regarding the sentiment shifters, their impact is exemplified by the following sentence: cum semel occideris et de te splendida Minos fecerit arbitria non Torquate genus non te facundia non te restituet pietas (`When you at last have died and Minos renders brillant judgement on your life, no Torquatus, not birth, not eloquence, not your devotion will bring you back.' - ode IV, 7). Here, the sentiment score calculated by the script is very positive (3) because it does not handle the frequent negations: however, the particle non should reverses the positive polarity of facundia `eloquence' and pietas `devotion'. This problem could be mitigated by modifying the script with rules that take into account negations and their focus. Regarding the zero-shot classification approach, the very low performances on Latin deserve further investigation. It is possible that the problem lies in the data used to build the pre-trained models: i.e., Wikipedia for mBERT and Commoncrawl for XML-RoBERTa. Both resources were developed by relying on automatic language detection engines and are highly noisy due to the presence of languages other than Latin and of terms related to modern times. An additional improvement may also come from using for fine-tuning an annotated in-domain corpus in a well-resource language, that is a corpus of annotated poems: unfortunately, the currently available corpora are not big enough for such purpose.