Identifying Visual Depictions of Animate Entities in Narrative Comics: An Annotation Study (2023)

Abstract: Animate entities in narrative comics stories are expressed through a number of visual representations across panels. Identifying these entities is necessary for recognizing characters and analysing narrative affordances unique to comics, and integrating these with linguistic reference annotation, however an annotation process for animate entity identification has not received adequate attention. This research explores methods for identifying animate entities visually in comics using annotation experiments. Two rounds of inter-annotator agreement experiments are run: the first asks annotators to outline areas on comic pages using a Polygon segmentation tool, and the second prompts annotators to assign each outlined entity’s animacy type to derive a quantitative measure of agreement. The first experiment results show that Polygon-based outlines successfully produce a qualitative measure of agreement; the second experiment supports that animacy status is best conceptualised as a graded, rather than binary, concept.

## introduction comics are a rich multi-modal medium for automatic discourse processing, yet empirical work investigating their narrative structures is still a nascent research area. current approaches include computational descriptions of narrative structures that are used to automatically generate comics from chat scripts @xcite , video game logs @xcite , or video @xcite . approaches applying linguistic methods includes visual narrative grammar, which categorizes panels based on their narrative function and describes grammar-like constraints distinguishing valid from invalid panel sequences @xcite , and deriving relationships between high-level narrative structures and pat-terns of low-level text and image parts @xcite . this research is part of a larger project that seeks to identify narrative affordances in comics by examining compositions of units such as panels, text, symbols, characters, backgrounds, etc. taking inspiration from annotation schemes for text narrative, we explore a method of annotation and assess the reliability with inter-annotator agreement experiments. this paper focuses on identifying animate entities in images. our previous work examined coreference agreement of characters, and we hope to link image and discourse referents in the text in future work. we hope this work contributes to a full-fledged annotation scheme that can be applied to future corpora which would contain both annotations in the non-text areas of comics, as we look at here, and annotation of the textual areas of comic pages, for a truly multi-modal approach to discourse referents. ## experiment 1: identifying animate entities using outlines on comic pages this first experiment tests a method for delineating animate entities according to reader judgments. annotators are prompted to outline areas directly onto comics pages where they believe shows a depiction of an animate being. having annotators draw on the page circumvents the researcher's assumptions about what should be considered animate, since the researcher is not pre-selecting potential candidates for annotators to judge. the areas outlined by each annotator are compared against one another to produce a qualitative measure of agreement. ## experiment 2: introducing an animacy hierarchy this experiment builds on the previous one by asking annotators to assign a hierarchical animacy type to pre-outlined entities on comic pages. experiment 1 implemented animacy identification through a coarse binary choice -outlined areas contain an animate entity according to the annotators judgment, while everything outside the outline does not. while the results of this technique gives initial insight into animacy indicators, the nature of disagreements can be further parsed using a quantitative metric. a simple hierarchy of animacy status is devised to assess whether agreement can be achieved by offering annotators several options. ## conclusion animate entities are an important component of defining characters and understanding broader narrative structures and affordances in comics. we explored methods of identifying animate entities visually through two rounds of annotation experiments: the first asked annotators to outline areas on comic pages with a polygon outlining tool, and the second attempted to quantify agreement by having annotators assign a hierarchical animacy category to each entity. results from the first experiment show that an outline-based agreement method, which imposes a binary concept of animacy, can qualitatively measure agreement between stories. in experiment 2, we develop a hierarchy of animacy types based on disagreements from the first experiment, with the results showing that a graded rather than categorical concept of animacy performs adequately for some stories. an ordinal scale may better capture edge cases where the line between human-like and animal-like behaviours; for instance, a crow shows human-like behaviours such as strategical planning. additionally, both studies suggested that purposeful animacy ambiguity is a valuable narrative tool, and should be accounted for in future developments of the annotation scheme.