Annotating Spatial Descriptions in Literary and Non-Literary Text (2025)

Abstract: Descriptions are a central component of literary texts, yet their systematic identification remains a challenge. This work suggests an approach to identifying sentences describing spatial conditions in literary text. It was developed iteratively on German literary text and extended to non-literary text to evaluate its applicability across textual domains. To assess the robustness of the method, we involved both humans and a selection of state-of-the-art Large Language Models (LLMs) in annotating a collection of sentences regarding their descriptiveness and spatiality. We compare the annotations across human annotators and between humans and LLMs. The main contributions of this paper are: (1) a set of annotation guidelines for identifying spatial descriptions in literary texts, (2) a curated dataset of almost 4,700 annotated sentences of which around 500 are spatial descriptions, produced through in-depth discussion and consensus among annotators, and (3) a pilot study of automating the task of spatial description annotation of German texts. We publish the codes and all human and LLM annotations for the public to be used for research purposes only.

## introduction literary and non-literary texts are full of descriptions that help readers see, hear, feel, smell, and even taste what is happening in a story or text, making the places and entities experiencable. while the analysis of literary text has become an important area of annotation studies, existing work typically targets narrative elements, such as characters or plot structure @xcite @xcite @xcite @xcite . in the domain of non-literary text, a lot of recent nlp work deals with multimodal image descriptions scraped from alt-texts on the web or collected via human annotations, cf. @xcite @xcite @xcite . however, to our knowledge, no tool or dataset distinguishes between descriptive and non-descriptive language and identifies descriptions in naturally occurring text. in this work, we present an approach to annotating and detecting descriptions in unimodal, literary, and non-literary text. to give our study a concrete target and domain, we focus on descriptions of space. since the 1990s, the concept of space has gained increasing attention in the cultural and social studies @xcite . in linguistics and nlp, the analysis of spatial language in text has received moderate but continuous attention. to date, existing work on annotations of spatial language mainly aimed at detecting mentions of spatial entities (named entity recognition) or other spatial concepts, like paths or trajectories @xcite . this work focuses on identifying sentences describing static space. the following sentence is an example of a spatial description in a story that works without naming any named spatial entities: auf dem zertretenen rasen zwischen haus und zaun, roh gezimmert, stand ein länglicher tisch mit bank und sesseln. 2 on the trampled lawn between the house and the fence, rough-hewn, was an oblong table with a bench and chairs. in literary texts in particular, such descriptions are a fundamental unit for creating a space of action and opening up a world to the reader by routing the narrative in a physical environment. despite the increasing interest in space and spatial descriptions, identifying them in a natural context-in our study, novels or travel reports-remains a challenge. the paper contributes to the broader goal of understanding spatial and descriptive language in various textual domains and improving its automatic detection. we propose a set of annotation guidelines to extract spatially descriptive sentences from literary and non-literary texts beyond self-evident cases. as examples we use the two german corpora kolimo @xcite and wikivoyage @xcite . based on samples extracted from these two corpora, we created a set of annotated sentences. to ensure that all annotators' perspectives are considered, we systematically discussed the cases of disagreement. a final label was assigned based on the mutual agreement of all annotators on a plausible classification. since human annotations are expensive and time-consuming, we also explore how to automate this annotation task. based on the manually annotated dataset, we test the ability of llms to identify spatial descriptions. in doing so, we aim to contribute to a more comprehensive understanding of spatial language processing. ## annotation procedure this section introduces the set-up of our annotation task, the procedures for guideline development and data curation, as the final annotation guidelines. ## spatial descriptions dataset our annotation work resulted in a dataset of spatial descriptions extracted from two fundamentally different german corpora of literary and non-literary texts: kolimo and wikivoyage. kolimo, the "corpus of literary modernism", has its focus on 19th century fiction @xcite . the copyright on these texts has expired, and they are public domain. kolimo is a convenient literary corpus because of its size and its availability in digital form with extensive metadata. as a non-literary counterpart, we chose wikivoyage, an online travel guide, as we expected to find many spatial descriptions there @xcite . the german version of wikivoyage is distributed under the cc by-sa 4.0 license. we developed our guidelines for spatial descriptions primarily based on kolimo. as a nonliterary counterpart that is highly different not only in genre but also in its time of origin, wikivoyage enables us to explore the extent to which the annotation scheme can be transferred to another domain. for annotating on the sentence level, the full texts required some preprocessing. we excluded texts shorter than 10 sentences, assuming that it is unlikely that authors will dedicate complete sentences to exclusively describe spatial surroundings in very short texts. we eliminated incomplete sentences and only included sentences that begin with a capital letter and end with a punctuation mark. bullet points, as they can be found in wikivoyage, inherently indicate the beginning of a sentence and, therefore, cannot appear within a sentence. moreover, only sentences with a minimum length of five words are considered for annotation. for better comparability between the two subsets, we pre-filtered the data. for each corpus, we determined the 10 most frequent non-named spatial entities (by lemma) @xcite based on a list of spatial entities generated by @xcite . inflected forms or spatial entities as part of compound words (as they are frequent in german) were taken into account as far as possible (see appendix a). we condensed the datasets to only sentences that contain one or more of the 10 most frequent spatial entities. pre-filtering definitely contributed to the proportion of spatial description among all annotated sentences, as reported in ## pilot study: automatic annotation our aim is to eventually have a larger dataset of spatial descriptions across different textual domains. to this end, we carried out a prompting experiment with llms to classify the literary and non-literary sentences in our dataset ( § 4) in a zero-shot setting. ## discussion natural language and especially literary text is inherently complex and often ambiguous. in our aim to identify spatial descriptions, we encountered several sources of disagreement. apart from uncertainties in the texts themselves, disagreement also resulted from unclear cases within the annotation guidelines and practical factors such as annotator error. in this section, we discuss the major reasons for annotator disagreement. unresolvable ambiguities within the data itself are the most prominent factor for disagreement. isolated sentences do not always provide clear evidence as to whether they constitute a spatial description according to our definition. (see, for instance, example 10 in appendix b: without context, our annotators could interpret it as a description of an actual, spatial scene as well as a pure abstraction and therefore not spatial. examples 11 and 12 were ambiguous for our annotators due to the polysemy of certain words.) @xcite results, on the other hand, suggest that an increased amount of context would not necessarily contribute to an increased iaa. we therefore assume that there will always be at least a certain level of disagreement between annotators simply due to the polyvalence of literary text @xcite . when the guidelines lack precision, however, it can result in fuzziness and different interpretations not of the text itself, but of the annotation scheme. @xcite claim that any fuzziness in the categorization must be minimized as much as possible. the inherent polyvalence of the texts does not justify ambiguity in the category definitions. on the other hand, it is generally not possible to formulate guidelines that unambiguously account for 100% of all cases @xcite . our attempts to make the guidelines as precise as possible resulted in a detailed seven-page document. @xcite warn of guidelines becoming too narrow and restrictive. they would be at risk of failing to capture the variability and polyvalence inherent to human language. in iteration 3 of our annotation, we had the most extensive list of guidelines in use. as a third and minor, but still a noticeable reason for an imperfect iaa was human errors @xcite . when processing a large number of individual sentences in succession, the cognitive effort of the annotators was considerable and could occasionally lead to the selection of incorrect categories. we argue that certain levels of disagreement are not only unavoidable but even indicative of the nuanced nature of descriptive and spatial language. we did not expect perfect agreement between the human annotators and even less between humans and llms. instead, the objective was to produce a curated dataset of spatial descriptions in which any ambiguity arises solely from legitimate differences in the interpretation of language, accounting for the subjectivity of the individual annotators @xcite . the annotation process provided valuable insights into how humans interpret descriptive and spatial language and how annotation guidelines mediate this interpretation. in general, we observe that the task of description annotation features a certain amount of subjectivity, resulting in label variation in our data. while traditional nlp paradigms aimed at eliminating human label variation as much as possible, recent work argues for embracing rather than excluding or ignoring it @xcite . by making the different iterations of our annotations and guidelines available, we also hope to contribute to this emerging line of research.