Interpretation of Sentiment Analysis in Aeschylus’s G reek Tragedy (2020)

Abstract: Recent advancements in NLP and machine learning have created unique challenges and opportunities for digital humanities research. In particular, there are ample opportunities for NLP and machine learning researchers to analyze data from literary texts and to broaden our understanding of human sentiment in classical Greek tragedy. In this paper, we will explore the challenges and benefits from the human and machine collaboration for sentiment analysis in Greek tragedy and address some open questions related to the collaborative annotation for the sentiments in literary texts. We focus primarily on (i) an analysis of the challenges in sentiment analysis tasks for humans and machines, and (ii) whether consistent annotation results are generated from the multiple human annotators and multiple machine annotators. For human annotators, we have used a survey-based approach with about 60 college students. We have selected three popular sentiment analysis tools for machine annotators, including VADER, CoreNLP’s sentiment annotator, and TextBlob. We have conducted a qualitative and quantitative evaluation and confirmed our observations on sentiments in Greek tragedy.

## introduction recent advancements in nlp and machine learning have created unique opportunities for digital humanities research. in particular, sentiment analysis toolkits provide a way to explore the representation of emotions in literary texts such as ancient greek tragedy. aristotle defined tragedy as a medium for bringing out emotions, especially pity and fear. greek tragedies express a plethora of emotions via the characters and their narratives. recent advancements in the nlp and machine learning make it possible to conduct a systematic analysis of these sentiments and emotions using computational tools. recent work has explored the differences between sentiments and emotions in greek tragedy and contemporary society. there have been two contrasting views on a universal emotion across time and space. on the one hand, kalimtzis used david cairn's school of thought that "cultures exhibit points of overlap that make them mutual intelligible," in other words, having "naive assumption of shared humanity" @xcite . in contrast, konstan endorses the opposing view that emotions observed in greeks of the classical period are different from the modern ones @xcite . in recent years, there has been scholarly work focused on comparing the contemporary population's emotional impact based on greek tragedy and horrific contemporary events @xcite , or even focusing on the role of emotions in ancient greek diplomatic practice @xcite . sentiment analysis could be used as a training step for machines to perform more complex tasks like emotion detection. however, sentiment analysis is not an easy task for a machine because of the multiple and often unpredictable variables applied to interpret a given sentence. typically, sentiment in context is this work is licensed under a creative commons attribution 4.0 international licence. licence details: http://creativecommons.org/licenses/by/4.0/ . an incredibly complex task for machines. a study @xcite presented a highly complex and dynamic system for reflecting the rich structure of human interaction and communication and identifying associated sentiments and topics by characterizing relationships explicitly. it demonstrated how these methods could be used to explore victor hugo's les misérables. in our study, sentiment analysis on greek tragedy was conducted using both social media trained sentiment analysis tools and human annotators. this work is an initial step in exploring promising research on the modern understanding of ancient emotion. furthermore, to conduct advanced sentiment analysis in machine learning, we need well-annotated data that can be used to teach machines about emotion in greek tragedy. the human-in-the-loop @xcite has received attention for the potential of human and machine collaboration. more reliable data can be collectively annotated for in-depth study in machine learning from this kind of collaboration. in this paper, we explore the challenges and benefits from the human and machine collaboration for sentiment annotation about sentiments in greek tragedy and address some open questions related to the collaborative annotation. we are particularly interested in analyzing why similar or different behaviors or opinions may be observed from the machine and human annotators-and what a comparison of human annotators and machine annotators can teach us about how humans and machines read emotion. we mainly focus on (i) an analysis of the challenges in sentiment analysis tasks for humans as well as machines and (ii) whether consistent annotation results are generated from multiple human annotators and multiple machine annotators. for this study, aeschylus's essays were obtained in tei conformant xml texts from the perseus digital library @xcite . we extracted sentences based on stratified sampling, as shown in ## experimental results and evaluation figure 1 : correlation testing these models for sentiment analysis are based on the lexical, grammatical, and syntactical conventions of model sentiment. expressing sentiment intensity is determined by rules that might differ from sentiments as they are expressed in the literary texts. our initial hypothesis is that human annotations of sentiments in greek tragedy will be similar to the machine annotations and validate the applicability of these toolkits for literary texts. this is determined as follows: first, machine annotators face similar challenges to those of human annotators. since machine annotators rely on a dictionary of word meanings, they can be consistently applied for sentiment analysis. however, some terms from greek tragedy may not be adequately determined by machine annotators. contemporary meanings in social media, for which the sentiment analysis tools were developed, may not align. second, machine annotators have their own discrete sentiment annotations. for example, the annotation ratings for vader, corenlp, and textblob vary. for simplicity, we have normalized them into five annotations, consistent with the human annotations ranging from extremely positive (1) to extremely negative (5). finally, since machine annotators cannot determine a sentiment annotation when the input is given with the surrounding context, it is difficult for multiple machine annotators to get a consistent annotation result. we used correlation tests to determine whether the values of quantitative variables change in conjunction with each approach. first, we computed the pairwise correlation coefficients for quantitative variables using the pearson coefficient (r) and represented them in a heatmap. second, we checked whether the variation between the sets of variables is monotonic (increasing or decreasing) or whether the data's underlying distribution is normal. correlation coefficients ranged from -1.00 to +1.00. a positive value indicates a positive correlation -one variable is increasing, and so does the other -while a negative value indicates a negative correlation -one variable is increasing and the other decreasing.