Mining the Past: A Comparative Study of Classical and Neural Topic Models on Historical Newspaper Archives (2025)

Abstract: Analyzing historical discourse in large-scale newspaper archives requires scalable and interpretable methods to uncover hidden themes. This study systematically evaluates topic modeling approaches for newspaper articles from 1955 to 2018, comparing probabilistic LDA, matrix factorization NMF, and neural-based models such as Top2Vec and BERTopic across various preprocessing strategies. We benchmark these methods on topic coherence, diversity, scalability, and interpretability. While LDA is commonly used in historical text analysis, our findings demonstrate that BERTopic, leveraging contextual embeddings, consistently outperforms classical models in all tested aspects, making it a more robust choice for large-scale textual corpora. Additionally, we highlight the trade-offs between preprocessing strategies and model performance, emphasizing the importance of tailored pipeline design. These insights advance the field of historical NLP, offering concrete guidance for historians and computational social scientists in selecting the most effective topic-modeling approach for analyzing digitized archives. Our code will be publicly available on GitHub.

## introduction digitized newspapers have become widely used in recent years, providing convenient access to extensive historical records. online platforms further support historians in efficiently identifying and analyzing primary and secondary sources @xcite . however, the vast amount of documents and information available presents a challenge for historians in terms of study, analysis, and interpretation. to address these challenges, natural language processing (nlp) methods are frequently employed to streamline the process. in our recent work, we present a novel approach for both extractive and abstractive summarization of historical texts @xcite . in this paper, we focus on topic modeling (tm) methods to automatically extract themes from historical newspaper archives, reducing the time historians would otherwise spend on manually categorizing and analyzing these contents. latent semantic analysis (lsa) @xcite ) laid the foundation for tm. building on this, the probabilistic framework known as probabilistic latent semantic analysis (plsa) @xcite was introduced. however, the development of latent dirichlet allocation (lda) @xcite represents a significant turning point in the field, providing a more sophisticated and effective probabilistic approach for uncovering latent topics within large-scale text corpora. another widely used technique is non-negative matrix factorization (nmf) @xcite , which employs matrix factorization technique by decomposing a term-document matrix into two lowdimensional, non-negative matrices representing words and documents. building on these foundational methods, many new approaches have emerged in recent years. the introduction of the transformer architecture @xcite revolutionized many nlp aspects and paved the way for the development of advanced neural-network models. since then, traditional tm techniques have been enhanced by neural-based methods that leverage contextual embeddings. among these, two widely used approaches are top2vec @xcite and @xcite . these models demonstrate promising performance in capturing contextual meaning and intricate patterns within textual data, significantly outperforming conventional methods. while lda and nmf have been widely applied across various fields, including historical research, neural topic models still remain underutilized in this domain. @xcite compared four topic mod-els on twitter posts, which are short texts, using qualitative evaluation. however, these findings cannot be directly applied to newspaper articles, as they often cover multiple topics within the same document. given the structured and in-depth nature of news articles, it is crucial to evaluate topic models, specifically in this context. to address the challenge of selecting the best topic-modeling approach for historical newspaper articles, we conduct a comprehensive empirical evaluation of classical and neural topic models on a large historical newspaper dataset. the main contributions of our work are as follows: by systematically analyzing various preprocessing methods, different embedding models, and model performance, we offer tailored recommendations for analyzing historical archives. to the best of our knowledge, this study is the first comprehensive comparison of four topic-modeling methods specifically applied to a large-scale historical news archive. ## related works this section reviews historical topic modeling, existing approaches, and future directions. classical topic modeling methods, such as lda and nmf, have been widely used in the historical domain for topic detection. lda @xcite is a probabilistic model that represents documents as topic mixtures and topics as word distributions, using inference algorithms to estimate these topic distributions. nmf @xcite is based on matrix decomposition, where the document-term matrix is factorized into two nonnegative matrices representing the topics and their corresponding word distributions. several works have employed these classical models in historical research. @xcite conducted a study to explore the development of ideas in the field of computational linguistics over time by applying lda to the acl anthology, covering the years 1978 to 2006. @xcite leveraged the lda topic model on the collection of digitized historical newspapers published in texas from 1829 to 2008. a very interesting study by @xcite provides a comprehensive overview of the history and application of tm within digital humanities, particularly in digital history from 2006 until 2012. only 23 historical tm studies were found during 2006-2012, and the majority were conducted to explore the topic methods users and its usage rather than using it for solving independent historical questions. another study by @xcite conducted lda-based tm to study the industrial and environmental history in scotland. @xcite also experimented with lda to the large archives of economic articles. zamiraylova and mitrofanova (2020) study leverages the nmf algorithm to automatically identify and analyze dynamic topics within a corpus of russian short stories from the first third of the 20th century, providing a deeper understanding of the thematic evolution in the russian literature. the recent studies in the historical domain continue to strongly rely on lda and nmf @xcite @xcite @xcite @xcite @xcite @xcite @xcite . neural topic modeling methods have gained popularity for capturing complex text relationships using deep learning. recent tm methods, such as top2vec @xcite and bertopic (grootendorst, 2022), leverage neural embeddings and clustering techniques to improve topic discovery, offering greater flexibility and coherence compared to classical methods. only very few studies have applied neural models in historical tm. @xcite proposed discourse atom topic modeling (datm), a novel method, that integrates probabilistic topic modeling with word embeddings applied to violent death narratives in the u.s. national violent death reporting system, revealing nuanced themes and gender biases. @xcite introduced the neural dynamic focused topic model (ndf-tm), which uses bernoulli random variables and outperforms classical dynamic topic models in tracking topic evolution across the un general debates, neurips papers, and acl anthology datasets. martinelli et al. (2024) compared two neural topic models, product-of-experts lda, and embedded topic model, against lda on a classical latin corpus. their evaluation found that neural models outperformed lda in both quantitative metrics and expert qualitative assessments. ginn and hulden (2024) compared dynamic topic models on 1,350 roman literature texts, finding that neural models aligned better with historical intuitions than classical models. shortcomings. classical topic models fall short in capturing historical text semantics, while neural models provide richer, context-aware representations. other domains have advanced by adopting neural topic models, which use deep learning techniques for more sophisticated and accurate topic representations. @xcite . the adoption of neural-based topic models in historical research remains limited. only a handful of studies have ventured into using neural approaches so far, thus leaving a significant gap in the methodological toolkit available for historians. this lag indicates a pressing need for the historical domain to embrace and experiment with neuralbased tm techniques. motivated by this prevalent gap, we empirically analyzed classical and neuralbased topic methods. specifically, we picked two classical models, lda and nmf, which are popu-larly used in the historical domain as baselines, and we compared them with the more recent top2vec and bertopic neural-based models. we tested all methods on a large collection of more than 148,000 historical newspaper articles centered around the themes of "nuclear power" and "nuclear safety" to evaluate their performance. ## methodology in this section, we outline the workflow used to conduct our study as presented in figure 1 . ## empirical results & analysis this section outlines the experimental setup, evaluation metrics, and empirical results, followed by quantitative and qualitative analyses. ## conclusions in this paper, we conducted a comprehensive evaluation of four topic-modeling techniques-lda, nmf, top2vec, and bertopic-in combination with three text-embedding models. while our experiments leverage hpc for large datasets, all tested methods remain effective on standard hardware for smaller datasets, ensuring accessibility and scalability across diverse computational settings. our experiments show that lda excels in topic diversity but struggles with coherence, while nmf generates more coherent topics but suffers from redundancy with a large number of topics. bertopic with a large sequence-length embedding model outperforms both, offering superior coherence, diversity, and the ability to handle longer texts without losing context. we recommend bertopic for large, heterogeneous datasets due to its balance of efficiency, coherence, and diversity, although careful preprocessing is necessary for models like smaller embeddings models. our empirical analysis provides clear guidance for digital humanities researchers and users in selecting the most appro-priate topic modeling method for their specific use cases, particularly when dealing with large datasets.