“Orpheus Came to His End by Being Struck by a Thunderbolt”: Annotating Events in Mythological Sequences (2023)

Abstract: The mythological domain has various ways of expressing events and background knowledge. Using data extracted according to the hylistic approach (Zgoll, 2019), we annotated a data set of 6315 sentences from various mythological contexts and geographical origins, like Ancient Greece and Rome or Mesopotamia, into four categories: single-point events (e.g. actions), durative-constant (background knowledge, continuous states), durative-initial, and durative-resultativ. This data is used to train a classifier, which is able to reliably distinguish event types.

## introduction in narratological terms, events have been defined as "constitutive features of narrativity" @xcite , the atomic building blocks of a story. an utterance is an event if it communicates a change of state, a "transformation", which is a fundamental property of any event. in order to produce a plot or a story, events need to follow a chronological or diegetical order, with events being subject to a change in time. succession and transformation are therefore key principles in a narrative @xcite . @xcite distinguished two types of event structures in texts: the surface structure, represented by verbal predicates, and the latent event structure, which refers to sub-events and their representations. according to @xcite , events are often conjoined with states, in the sense that a source state s occurs before the transition into a target state s', triggered by an event (or series of events) e. in narrative annotation studies, distinctions between events and states are most commonly attributed to the eventfulness of the predicate. the focus on the question what constitutes an event is very much on the question "who does what to whom?" this works presents the annotation efforts to classify different types of events in the mythological and religious domain. textual sources from those domains often do not narrate plots in a straightforward manner. instead, they use stylistic devices, like prolepses, to transport their narrative, which can make automatic extraction and event labelling challenging. for this study, different types of events and their chronological order have been extracted by domain experts from the fields of ancient near eastern studies, religious studies and classics from a large variety of sources. for each source, a sequence of events and background information was manually extracted based on the original, e.g. in ancient greek or sumerian, where available. those sequences were derived according to the hylistic approach @xcite from the narrative domains of mythological and religous studies @xcite . each sequence corresponds to one variant of a (mythological) plot in the respective source. the context window of the myth variant, i.e. which passages of the source refer to a mythological plot, is identified by the domain expert. hence, the text passages that correspond to the sequences, as well as the sequences themselves, differ in length. the sequences can be used for comparatistic tasks, such as the comparison of narrative plots or background information, e.g. the characterization of entities. the distribution of disciplines from which the sources are taken are presented in figure 2 . the narrative sequences contain practically no discourse markers, and are comprised of individual statements (hylemes) which are strictly in present tense. the hylemes contain events, including stative events or states in the chronological (narrative) order, not the diegetic order. as an example, we use the quote 'orpheus came to his end by being struck by a thunderbolt' 1 . from this sentence, the following short sequence of statements (hyleme sequence) can be manually extracted: the task of this work is to annotate the event types of each individual statement (hyleme). the data is in german, examples have been translated by the author for this paper, where necessary. this paper is structured as follows: section 2 frames the work into context of similar annotation efforts. section 3 compares the categorisation of events used in this work with previous annotation efforts by @xcite . the data set used for this study is described in section 4. in section 5, we describe the annotation effort and its results. we present a simple classifier to determine the event types, which we describe in section 6. finally, the paper ends in a discussion in section 7. ## related works our work is situated in the context of mythological research, but has potential for application in other domains. on the linguistic level, it is related to the study of lexical aspect (or aktionsart) and the situation entity (se) annotation task. @xcite label se types from clauses in a supervised sequence modelling task using features of the main verb, its main referent, and the clause itself. they report good results across different genres. in an earlier study, @xcite annotated clausal aspect for automatically recognising whether a clause describes a habitual, episodic, or static phenomenon. @xcite sucessfully identified temporal aspect (telicity and duration) in english and french data sets using a transformer approach. furthermore, there are a number of practical approaches which attempt to define and narrow the narrative concept of events and their representation. @xcite introduce an approach to use unsupervised learning of event chains centered around an event protagonist. they train a temporal classifier to produce a temporally ordered narrative chain. in a subsequent study, they present the concept of narrative schemas, as "coherent se-1 pausanias, description of greece quences or sets of events" @xcite . by applying an unsupervised learning approach, they add semantic roles to the argument structure of their event chains. multiple events chains are then combined into a narrative schema. timeml @xcite ) is a markup language designed based on xml which provides a standardized way of annotating temporal expressions and events in text, including the temporal relationships between events. it is used for the annotation of temporal and event information. four automatic timeml annotation systems have been evaluated by @xcite . reiter (2015) compared the annotation of narrative segments performed through crowd-sourcing, by student annotators and summary annotations. kwong (2011) annotated a corpus of fables regarding their structural and semantic properties, including temporal information. events that are part of a script, such as 'baking a cake', have been automatically mapped to narrative texts by @xcite . events and event types in narrative plots have been studied by @xcite . they operationalize the concepts of narrativity and tellability as discourse phenomena. they use spans of text defined by finite verbs as annotation units. guis' and vauth's concepts of states and events are probably closest to those of the hylistic theory presented by @xcite . therefore, we will compare the two annotation approaches in more detail in the next section. ## event categories the narrative event model of @xcite and @xcite uses four categories of events: change of state, process events, stative events, and non-events. the basis of their event representation is the finite verb in 'minimal sentences', i.e. all tokens that are assigned to the verb. in contrast, the categories used for hylistic analysis are: single-point (punctual), durative-constant, durative-initial and durative-resultative. we classify a statement (hyleme) into one of these four categories, but the value is of course mainly associated with the verb. in both theories, each annotation unit has one finite verb. figure 1 illustrates the difference between four hyleme types. single-point hylemes are true at one point during the narrative sequence extracted from the source. this includes active actions, passive experiences, reactions, perceptions or feelings. the single-point event has its beginning and end during the sequence. however, that does not necessarily indicate an event with a short duration. durative hylemes hold true for a part of the sequence or over the course of the entire sequence. there are three sub-types: durativeconstant, durative-initial, and durative-resultative. durative-constant hylemes are always true, e.g. "orpheus is the son of oeagrus." they often communicate background knowledge about the narrative. additionally, certain 1n/ns statements 2 @xcite , e.g. "hades works the sails" are also considered durative-constant. there are two types of states which are true over a part of the sequence, but change their value at some point. durative-initial hylemes are true at the beginning of the sequence. durative-resultative hylemes are statements that become true at some point during the sequence (e.g. 'orpheus is dead.') and remain true for the rest of the sequence. in the mythological domain, these context-sensitive hylemes often connect contexts and plots. hyleme sequences follow a relative temporal order, without discourse markers. guis and vauth's category 'change of state', used for the first example sentence in the second example sentence "found he himself in his bed into a monstrous insect-like creature transformed" would be annotated as single-point statement according to hylistic theory, because the predicate "found" implies that he realises he has been transformed into a bug exactly once 3 during 2 "narrating one time what happened n times" 3 afterwards he knows that he is a bug. (durative-constant) the course of the narrative. however, a hylistic analysis of the plot would necessarily include a statement like "gregor samsa is a human transformed into a bug", which would be annotated as durative-constant. this statement does not need to be explicitly stated in the text, it can be implied. the sentence "his room lay quietly between the four well-known walls" demonstrates where the main difference between the two theories lie: according to @xcite , this sentence is annotated as a stative-event. while the hylistic theory @xcite also recognises that this is an ongoing state, it distinguishes between types of ongoing states. hylemes that are valid at the beginning, but change during the course of the narrative are categorised as durative-initial, e.g. eurydice is alive. hylemes that are the result of an event, e.g. a snake bites @xmath0, are durative-resultative. thirdly, there are hylemes that are true over the entire course of the narrative, e.g. eurydice is orpheus' wife. those statements are durative-constant. they communicate the background knowledge that is the basis of a narrative, e.g. information about characters, their relations between each other and properties of the world in which a (mythological) story takes place. in order to determine the hylistic event category of the third sentence, therefore, we need to establish if the quietness of the room is a) the result of something that happened previously, or b) the initial state that is changed later-on, e.g. by someone barging in, or c) a general characterisation of the room. durative-resultative statements are often preceded by a single-point statement, which corresponds to a change of state event according to @xcite . however, occasionally durativeresultative statements are the result of the entire narrative, e.g. "no one can solve this incantation" is the result of the entire narrative of the invocation ms 2353 @xcite @xcite . non-events are not represented in hylistic theory, because they do not contain plot relevant information. non-events contain mainly conditional, subjunctive, or modalised statements @xcite . ## data as explained in the previous section, the event definition used in this paper is different from the ones mentioned in section 2. furthermore, event stative events durative(-constant, the four well-known walls" -initial, or -resultative dependent on context) "she would have closed the door non-events na to the apartment" statements (hylemes) are not derived directly from the textual representation in a source. mythological plots and descriptions of background knowledge are often not told in a straightforward manner. rather, they allude to related aspects of similar myths, and use comparisons, context and intertextuality in ways that makes the interpretation of what exactly happens in a myth variant hard to understand for laymen and even harder to process using nlp tools. even the order of events is sometimes difficult to establish, as the following example illustrates: (1) "but orpheus, son of oeagrus, [they sent back 4 with failure from hades], [showing 3 him only a wraith] [of the woman for whom he came 2 ]; [her real self they would not bestow 3 ], [for he was accounted to have gone upon a coward's quest 1 ], ..." 4 we can see that the sequential order of events is different from the order presented in the source. chronologically, orpheus first goes on a coward's 4 plato symp. the next example will illustrate how main plot events in classical sources are communicated merely by allusion. both examples show that extracting information from the texts is a challenging task that needs to be guided by informed scholars. this issue severely magnifies if we do not consider the well documented classical domain, but extend studies to fields like ancient near eastern studies, where sources are often scarce, and their supporting material (e.g. cuneiform on stone tablets) can be damaged or difficult to read. therefore, the context-window, the plot inherent events and background knowledge presented in the 228 sources have been extracted manually. each source is presented in one sequence of event statements, so called hyleme sequence @xcite . the hylemes were originally not annotated with their state or event types. however, in order to process the sequences for further study using nlp methods, e.g. measuring the similarity of plots or aligning variants of the same myth, the annotation of single-point events, and durative statements was needed. each hyleme sequence describes the plot of one myth variant and related background information. the statements usually do not contain fixed or relative temporal expressions, or relations such as before or after. instead, the succession of events is expressed through the sequential order. the annotated data is a set of 6315 hylemes and their assigned category. it is not, as discussed above, an annotation of concurrent text from the sources, but sequences describing the plot that were extracted manually. the statements themselves are usually short, concise sentences in german, consisting of only main clauses, containing one finite verb in present tense and active voice (where possible). co-references are widely avoided. instead, each statements contains the resolved arguments, which are repeated in the subsequent statements, even if they are only communicated by co-references in the text. one sentence in a source can translate to multiple statements, e.g. "orpheus is the son of oeagrus.", "the gods send orpheus back from hades as a failure.",... aspects which are alluded but can be safely determined by the informed scholar (e.g. orpheus' wife's name is eurydice) are added in square brackets. those implications can be part of the statement, e.g. a name, or an entire statement. for instance, in the example sentences from kafka's metamorphosis, the first statement "gregor samsa one morning from uneasy dreams awoke" would be preceded by a statement like "gregor samsa is sleeping" in a hylistic analysis. ## annotation the data set was annotated by six annotators. since durative-initial and durative-resultative statements are context-sensitive, annotators always processed the entire sequence. each narrative sequence was annotated twice. all annotators had previous experience with the hylistic theory. additionally, they were trained in an initial annotation meeting. each annotator was given a set of sequences, which were annotated individually and discussed by the group afterwards. annotators were also given a set of guiding questions and example statements to help them chose the right event category where in doubt. the guiding questions were presented in a flowchart. additionally, annotators with explicit knowledge in the field, e.g. classics, were also asked to check the original sources for guidance where in doubt. for example, the english statement "orpheus brings back the dead (from the netherworld)" can be interpreted as single-point or durative-constant. through the original greek source, it can be determined that it should be annotated as durativeconstant, because the imperfect form (ἀνῆγεν) is used @xcite . in a second meeting, questions that arose during the annotation process were discussed. items were annotated in 11 different pairings, with varying first and second annotators. in all but one cases, the inter-annotator agreement for the annotation task ranges from substantial (κ 0.61-0.80) to almost perfect agreement (κ 0.81-0.99). the agreement is reported in in cases where the first and second annotator disagreed, the gold standard was derived by discussion in a separate meeting, or following the judgement of the annotator whose discipline the sequence belongs to. performance of annotators against gold standard, and total number of annotated items are reported in the final gold labels are an important foundation for the next analyses, e.g. plot comparison and alignment or the comparison of background information in the individual sources. the distribution of the gold-standard labels is shown in figure 3 . the majority of the data consists of singlepoint statements, of the durative statements, the durative-constant hylemes are the largest group. three hylemes had to be excluded from the data, because their types could not be determined (e.g. the statement "the kur-ĝara and gala-tur ...?" has a missing predicate due to the source not being properly readable). ## classifier based on the gold labels of the annotation as described in the previous section, two event type classifiers were trained. 6 the resulting models can be used to pre-classify new statements, and to classify statements in future data sets that can be used for comparison, e.g. including movie adaptations of mythological narratives. the separation of the data into durative and single-point statements is an important first step for the subsequent analyses of the narrative sequences, since single-point statements correspond to events, whereas durative hylemes correspond to descriptions of background knowledge. the task to automatically classify event types is not trivial. especially, automatically distinguishing the three types of durative hylemes is challenging. this is due to multiple reasons. firstly, the three classes are unbalanced, with more durative-constant hylemes, and very few durativeinitial hylemes. additionally, durative-initial or -resultative hylemes can be quite similar to durativeconstant hylemes in terms of vocabulary and grammatical structure. as discussed above, their value is often context-sensitive. for the classification task, a multinomial naive bayes model was selected. for that purpose, the data set was split into a training and test set with a split of 75 %-25 %. the hyper-parameters were selected by performing a grid search. in particular, the grid search established whether the feature vector is best constructed using a bag-of-words or tf-idf vectorizer. as a result, the hyper-parameters were set as: laplace smoothing parameter α = 0.01, bag-ofwords features, and an n-gram range of 3. firstly, we analyze the results for binary classes single-point and durative, which combines durative-initial, durative-constant, and durativeresultative statements. for that purpose, all three labels were subsumed under the coarse class durative for training. the binary classifier performs well on single-point hylemes, and reasonably on durative hylemes. the performance of the classifier is reported in secondly, we investigate how the classifier performs if trained on just the different types durative hylemes. for that purpose, all single-point hylemes were removed from the training and test set. the majority of the test set consists of durative-constant hylemes (69 %) and durative-resultative hylemes (24 %). the results are reported in lastly, we present the classifier for the classification of fine-grained classes. it was trained on the entire training and test set including fine-grained durative classes. a second classifier combining the first two models (binary and durative-only) in two steps was trained but did not improve results. ## discussion in order to annotate event types for the mythological and religious domains from the source, the sequence of events and background information has to be extracted. automatically extracting these events from can be challenging, as demonstrated in the examples in section 4. therefore, the sequences of statements describing events and states from the sources was achieved manually. subsequently, we present annotations based on the hylistic theory @xcite , which was developed specifically for the mythological domain, but can be easily applied to other types of narrative as well. the data includes over 6300 statements from 228 narrative sequences. the statements have been annotated into four categories. singlepoint statements, communicating events, durativeconstant (background information), durative-initial prec. recall f1 support d.-initial 0.50 0.17 0.25 30 d.-constant 0.72 0.67 0.69 294 d.-resultative 0.55 0.45 0.49 103 single-point 0.90 0.95 0.93 1151 and durative-resultative, which hylemes indicate that their truth value changes during the course of the sequence. after training the annotators, an overall satisfying inter-annotator agreement κ was reached. the main weakness of the presented approach is that the event categories are not assigned directly to the text. this is due to the original source material being extremely diverse in form, language, and genre. instead, the labels are assigned to the hyleme sequences which require significant manual effort and knowledge of the original material. durative labels, especially durative-initial and durative-resultative, are context-sensitive. the value of a statement has to be assessed within the context of the narrative sequence. since two identical statements can have different labels in different contexts, the classification task is particularly challenging. this is the case especially if the label depends not only on a single preceding statement (e.g. eurydice dies. → eurydice is dead.), but on the entire sequence (e.g. nobody can solve this invocation.) when hyleme sequences are extracted from modern texts in well-resourced languages, such as german or english, the manual effort could be alleviated by employing nlp methods, such as named entity recognition or semantic role labelling. with a larger number of texts and corresponding sequences, it would also be possible to automatically identify candidate statements from text. the gold standard data represents the actual distribution of labels, i.e. single-point statements (actions) are more prevalent than durative statements. hence, the final data set is skewed which explains the performance of the classifier. in this work, a simple naive bayes classifier was implemented for demonstration purposes. a more sophisticated model, e.g. following a multi-lingual transformer approach @xcite , would potentially deliver better results. in future studies, the plots of mythological and religious narrative can now be studied and com-pared using nlp and alignment techniques on sequences of single-point statements. the background information in durative-constant can be included, or processed separately to represent the narrative-inherent background knowledge.