Tell, Don’t Show: Leveraging Language Models’ Abstractive Retellings to Model Literary Themes (2025)

Abstract: Conventional bag-of-words approaches for topic modeling, like latent Dirichlet allocation (LDA), struggle with literary text. Literature challenges lexical methods because narrative language focuses on immersive sensory details instead of abstractive description or exposition: writers are advised to *show, don’t tell*. We propose Retell, a simple, accessible topic modeling approach for literature. Here, we prompt resource-efficient, generative language models (LMs) to *tell* what passages *show*, thereby translating narratives’ surface forms into higher-level concepts and themes. By running LDA on LMs’ retellings of passages, we can obtain more precise and informative topics than by running LDA alone or by directly asking LMs to list topics. To investigate the potential of our method for cultural analytics, we compare our method’s outputs to expert-guided annotations in a case study on racial/cultural identity in high school English language arts books.

## introduction a common task in text analysis is the identification and exploration of latent themes or topics across documents, and unsupervised topic modeling has sustained its popularity in cultural analytics @xcite . latent dirichlet allocation (lda) remains popular @xcite @xcite , though the strength of language models (lms) have raised new possibilities (e.g. @xcite . while the rise of high-performing lms in natural language processing (nlp) has opened new avenues for cultural analytics research, it has also raised barriers for access, especially since researchers in the humanities may be constrained by api costs and compute resources. given this interdisciplinary context, we examine how one may perform topic modeling on literary passages with "small", resource-efficient lms. we use these lms to address a key challenge of computational literary analysis: literature often contains low-level depictions of characters' actions, thoughts, and dialogue, and so higher-level themes may evade classic text analysis tools reliant on lexical surface forms. this challenge of identifying patterns across documents is especially conspicuous for literary text: the golden rule of narrative writing advises authors to show, don't tell @xcite . that is, good storytelling should allow readers to experience a narrative through low-level sensory detail instead of high-level abstractive exposition. to address this issue, we ask lms to tell us what book passages show to better surface topics and themes shared across them. we propose a novel yet simple approach for running topic modeling on literary texts: instead of applying lda on the original texts, we apply it on lms' abstractive retellings of them (figure 1 ). we show that the topics we then obtain are more useful for identifying crosscutting themes than topics outputted by lda alone and than those obtained by directly requesting topic labels from the same lms ( §5). aside from evaluating our approach across a variety of topics, we closely examine its potential for supporting cultural analytics research in a case study. we apply our method on books taught in american high schools, and work with in-domain experts to identify passages that mention and/or discuss racial/cultural identity. our approach is able to predict passages' topic distributions in a manner that corresponds with human annotators' codes, and identifies more topics and passages of interest than baseline methods. overall, our conceptual framework of abstractive retelling holds promise for literary content analysis. we release our code at https://github.com/lucy3/tell_dont_show . recently, approaches that directly elicit topic labels from lms have emerged @xcite @xcite , though these frameworks often involve an intricate chain of prompting steps. in contrast, our proposed improvement over traditional topic modeling requires only one prompt. our method is most similar to prior work asking lms to describe documents in a manner that surfaces information that is implicit in them @xcite @xcite . given our focus on literature, our work also contributes to computational humanities @xcite . at this junction, computational text analysis supports "distant reading" @xcite , though some recent work has also examined the use of lms for close, interpretative reading @xcite . topic models have been applied to a range of textual forms, including books @xcite @xcite , historical newspapers @xcite , folklore @xcite , and poetry @xcite . topic modeling's appeal in cultural analytics stems from its potential to identify content of interest and relate texts to each other @xcite . our evaluation approach and case study also consider these use cases ( §5, §6). ## methods our task is to inductively discover latent topics across literary passages. we experiment with instruction-tuned lms designed to be resourceefficient within various model families: gpt-4o mini, llama 3.1 8b, phi-3.5-mini (3.8b), and gemma 2 2b. ## evaluation setup we evaluate the general use of our proposed topic modeling method in two ways. first, we obtain preexisting literary themes/tags from online resources, and use them to efficiently evaluate topic models' performance across passage sets spanning a wide range of topics ( §4.1). second, we conduct human evaluation of topics outputted at the passage-level, using a setup similar to a classic topic intrusion test ( §4.2). overall, our evaluation setup in this section aims to broadly assess the interpretive utility of topic predictions. ## evaluation results retell's most prominently predicted topics for passages that share the same gold label demonstrate our approach's ability to surface cross-cutting themes in literary texts ( retell produces topics that crowdworkers often judge to be semantically related to passage sets' gold labels ( still, there is some variation in how different models respond to different verbs. for instance, gpt-4o mini's summaries and descriptions outperform its paraphrases, while gemma 2 2b's retell variants do not share the same pattern, and its retell-paraphrase actually performs well ( regardless of models' idiosyncrasies, our three variants of retell consistently outperform the baselines default lda and topicgpt-lite ( human ratings of passage-level topic relevance provide additional support that retell is a competitive alternative to topicgpt-lite ( across two forms of evaluation, we show that our proposed approach, retell, is capable of identifying cross-cutting themes in literary passages. ## case study: race in ela books now that we've evaluated our approach across a variety of lms and topics, we conduct a deepdive demonstration and analysis of its ability to support distant reading within a specific use case. a common first-order task for scholars working with literature is identifying passages of interest for closer reading and analysis @xcite . bottom-up tagging of literary collections with topics can also support search and discovery @xcite . here, we investigate the extent to which topic modeling can surface passages of interest for researchers. specifically, we examine whether predicted topics align with expert-guided, theory-driven human annotations. in particular, of ongoing interest in education and social psychology is the question of how race and racism is taught, or could be taught, to students @xcite @xcite . within the context of english language arts (ela), or courses that teach reading and writing skills, researchers may investigate how literature depicts race (e.g. @xcite , how teachers contextualize these depictions (e.g. @xcite , and how literature and pedagogical practice may combine to affect students' understanding of themselves and others (e.g. @xcite . to conduct these types of studies, it may be useful to identify literary passages within curricular collections that substantively engage with topics regarding racial/cultural identity. in this case study, we run topic modeling on 1,645 human-annotated passages drawn from books associated with ela courses in the u.s. we supplement this set with 19.8k additional passages stratified sampled across these books, to yield robust topic estimates. we compare outputs produced by retell-summarize (@xmath3, topicgpt-lite, and default lda (@xmath4, with the former two run with an open lm, llama 3.1 8b. 4 by running three different methods juxtaposed against passage-level human annotations, we hope to illustrate how they may differ in practice. ## conclusion we propose a conceptually intuitive "abstractive retelling" approach for topic modeling of literature that pairs language models with classic lda. our proposed approach allows lda to go beyond lexical-level sensory details in narrative text, and offers a competitive alternative to both lda alone and directly generating topic labels, especially for resource-efficient, "small" lms. we also examine our method's potential for search and exploration of literary collections in a case study on race in ela books. though retell has room for improvement, it holds potential for supporting content analyses of literary collections at scale. overall, we hope that the simplicity of our method encourages uptake in research around lms' affordances and caveats in cultural analytics.