Automating Violence Detection and Categorization from Ancient Texts (2025)

Abstract: Violence descriptions in literature offer valuable insights for a wide range of research in the humanities. For historians, depictions of violence are of special interest for analyzing the societal dynamics surrounding large wars and individual conflicts of influential people. Harvesting data for violence research manually is laborious and time-consuming. This study is the first one to evaluate the effectiveness of large language models (LLMs) in identifying violence in ancient texts and categorizing it across multiple dimensions. Our experiments identify LLMs as a valuable tool to scale up the accurate analysis of historical texts and show the effect of fine-tuning and data augmentation, yielding an F1-score of up to 0.93 for violence detection and 0.86 for fine-grained violence categorization.

## introduction violence has been a defining element in human history, influencing cultural values, political structures, and social norms @xcite @xcite . understanding its role in shaping ancient civilizations provides valuable insights into societal evolution, power dynamics, and conflict resolution @xcite @xcite . to analyze historical texts for information on violent events, historians have traditionally relied on manual analysis, reading, and annotating vast amounts of text. while manual annotation remains a gold standard for nuanced interpretations, time and labor required for the sheer volume of ancient texts and their linguistic complexities make this approach intractable for exhaustive collections of ancient manuscripts. the rapid growth of digital archives and historical corpora underscores the need for automated methods to assist historians in extracting information more efficiently. large language models (llms), such as bert @xcite , roberta @xcite , and gpt @xcite , have successfully been applied to a wide range of classification tasks, also for scaling annotation of historical texts @xcite . so far, they have not been used to classify text passages denoting violent events. our research bridges the gap between the hermeneutical processes of historical analysis and the computational methods of natural language processing. we develop and evaluate methodologies that automate the annotation of violence in ancient texts while preserving the depth of understanding traditionally achieved through manual methods. as our gold standard, we use the manually curated eris database @xcite foot_0 , a large digital collection of violent events from ancient literature. we first identify the violent passages contained in eris within their original texts using classifiers based on llms. then, we further reproduce some more fine-grained annotations from eris, categorizing the violent passages across multiple dimensions: level of violence, contextual background, underlying motives, and long-term consequences. the results of our study show that llms offer a promising solution for extracting violence data. they can expedite the identification of violent events and the extraction of contextual information from ancient texts. with accurate results for a range of classification tasks around violence, llms can complement the expertise of historians, allowing them to focus on deeper interpretative tasks rather than the extensive and time-consuming data processing typically required. in the following, we first give an overview of related work (sec. 2) before we introduce our dataset and methodology (sec. 3). we then present our results (sec. 4) and discuss their implications (sec. 5) before we conclude with a short summary and ideas for future work @xcite . code and data are provided as supplementary material foot_1 . ## background & related work this section provides some background on violence research in history and the digital humanities (sec. 2.1). we then introduce large language models and discuss related work concerning llms for classification and annotation support (sec. 2.2). ## data and experimental setup in this section, we explain eris as the basis for our experiments (sec. 3.1), how we set up the experiments for violence detection (sec. 3.2) and violence categorization (sec. 3.3), and introduce the evaluation metrics used for both tasks (sec. 3.4). ## results in this section we provide our results for violence detection (sec. 4.1) and violence categorization (sec. 4.2). ## discussion the experiments demonstrated the potential of finetuned large language models (llms) in detecting and classifying violence in ancient texts. our evaluation demonstrates the models' strengths in violence classification, with an f1-score of up to 0.93. in manual classification recall is often the problem due to implicit or symbolic violence, ambiguous wording, and a bias toward precision, leading to missed instances. our finetuned and augmented models achieve a high recall, showing that llms can mitigate blind spots that humans miss. however, challenges like class imbalance, conceptual overlap, and abstract categories in multi-class tasks revealed areas for improvement. for violence categorization, our approach excelled in well-represented and concrete categories, such as "victory" and "destruction," but struggled with abstract or underrepresented categories like "intrapersonal violence" or "exile". conceptual overlaps, such as between "political" and "tactical" motives, also led to misclassifications. from the perspective of historians, choosing between fine-tuned models, tools like chatgpt, or manual annotation depends on specific project needs. we provide an overview over the specific features to be considered for applying fine-tuned llms and chatgpt (either via user interface or via api) in inference speed varies between fine-tuned models and api-based solutions. fine-tuned models offer lower latency but require dedicated hardware, while api-based models provide scalability but introduce network latency and rate limits. finetuning is preferable for low-latency applications, while apis offer scalability and ease of use. ongoing model maintenance refers to the continuous process of monitoring, updating, and retraining fine-tuned llms to maintain their performance and adapt to evolving data distributions or task requirements. when practitioners fine-tune their own models, they bear the responsibility for performance monitoring, infrastructure management, and regular model updates to ensure accuracy and relevance over time. ethical and bias considerations differ significantly between fine-tuned llms and api-based solutions. pre-trained apis are typically premoderated, incorporating safeguards to filter harmful or biased outputs. on the other hand, fine-tuned models require custom mitigation strategies @xcite , which can either reduce or amplify biases, depending on dataset quality and training methods. fine-tuning allows for domain-specific alignment but poses risks if ethical oversight is inadequate. the implications of this research extend beyond ancient texts, offering valuable insights for analyzing contemporary violence depictions, addressing modern datasets such as media reports, social media content, or legal documents. adapting the models to contemporary datasets would require adjustments to account for different linguistic styles, cultural contexts, and evolving definitions of violence, presenting an exciting avenue for interdisciplinary research. a significant gap lies in automating the identification of abstract or highly contextual categories, such as psychological impacts or symbolic violence. achieving this would require expanding datasets, understanding abstractions in llms @xcite , incorporating knowledge bases @xcite , and exploring advanced techniques like retrieval augmented generation (rag) @xcite . developing dynamic models that can learn from continuous expert feedback through techniques like reinforcement learning from human feedback (rlhf) could also bridge this gap @xcite . ## conclusion in this work, we proposed a framework for automating the classification and categorization of violent ancient texts using llms. our two main contributions are the development of models capable of accurately classifying violent sentences, and employing these models to automate the process of fine-grained violence categorization. in both cases, we showed the effect of fine-tuning the models. for violence detection, we also showed that data augmentation drastically enhances recall, which is the most important measure for supporting manual annotation. our results can enable historians to accomplish tasks that previously required months or years in minutes. to the best of our knowledge, we are also the first to utilize the openai api to classify violent ancient historical texts and compare its performance against other pre-trained models. our findings underscore the potential of llms to automate labor-intensive tasks and pave the way for large-scale text analysis in historical research. while fine-tuned llms provide structured and efficient classification, chatgpt remains useful for exploratory tasks, and manual annotation retains its importance in complex interpretative work. challenges remain, particularly with underrepresented classes and computational constraints. exploring larger models could enhance contextual understanding while maintaining runtime performance. future work in close collaboration with historians could help resolve ambiguous cases that even human experts find difficult to classify. a hybrid approach integrating automated classification with expert validation would maximize both efficiency and accuracy. additionally, incorporating surrounding textual context instead of analyzing passages in isolation could further enhance classification performance. our methods also offer potential for extending the eris database to annotate and include texts from more recent historical periods. adapting the models to contemporary datasets would require adjustments for linguistic style, cultural contexts, and evolving definitions of violence, presenting exciting opportunities for interdisciplinary research.