Dynamic Topic Modeling with BERTopic | by Sejal Dua | Towards Data Science

This marks the last article of a three-part investigation to better understand my Medium suggested reads by way of an NLP technique called topic modeling. In this article, I will introduce what I believe to be the most powerful topic modeling algorithm in the field today: BERTopic. I will also attempt to illustrate how temporally-aware interactive visualizations can comprise vast amounts of information without needing to manually inspect any documents or topics at a granular level.
Dynamic topic modeling, or the ability to monitor how the anatomy of each topic has evolved over time, is a robust and sophisticated approach to understanding a large corpus. My primary goal of this article is to highlight BERTopic’s capabilities in juxtaposition with the pain points of wordclouds that were previously discussed. I hope to prove that topic modeling can be accessible to not-so-technical people and that it is worthwhile to resist the tempting chokehold that wordclouds seem to have on the NLP space. Disclaimer: I’m very much guilty of making wordclouds to inspect whatever corpus I am working with, but I hope to discover more informative and mathematically-grounded alternatives.
BERTopic is a topic modeling technique that leverages BERT embeddings and c-TF-IDF to create dense clusters allowing for easily interpretable topics whilst keeping important words in the topic descriptions.
It was written by Maarten Grootendorst in 2020 and has steadily been garnering traction ever since. The two greatest advantages to BERTopic are arguably its straight forward out-of-the-box usability and its novel interactive visualization methods. Having an overall picture of the topics that have been learned by the model allows us to generate an internal perception of the model’s quality and the most notable themes encapsulated in the corpus.
The package can be installed via pypi:
pip install bertopic
If you anticipate that your project will make use of the visualization options included with the BERTopic package, install it as follows:
pip install bertopic[visualization]
Embed Documents: Extract document embeddings with Sentence Transformers. Since the data we are working with are article titles, we will need to obtain sentence embeddings, which BERTopic lets us do conveniently, by employing its default sentence transformer model paraphrase-MiniLM-L6-v2.
Cluster Documents: Create groups of similar documents with UMAP (to reduce the dimensionality of embeddings) and HDBSCAN (to identify and cluster semantically similar documents)
Create Topic Representation: Extract and reduce topics with c-TF-IDF (class-based term frequency, inverse document frequency). If you are unfamiliar with TF-IDF in the first place, all you need to know in order to generally grasp what is going on here is one thing: it allows for comparing the importance of words between documents by computing the frequency of a word in a given document and also the measure of how prevalent the word is in the entire corpus. Now, if we instead treat all documents in a single cluster as a single document and then perform TF-IDF, the result would be importance scores for words within a cluster. The more important words are within a cluster, the more representative they are of that topic. Therefore, we can obtain keyword-based descriptions for each topic! This is super powerful when it comes to inferring meaning from the groupings yielded by any unsupervised clustering technique.
For more information and theoretical backing pertaining to these three algorithmic steps, please consult the author’s comprehensive guide for in-depth explanations.
To create a BERTopic object in Python and move onto the fun stuff (dynamic topic modeling), we just need our preprocessed list of documents. After loading in the data with pd.read_csv(), we can either write some lambda apply functions to preprocess our textual data:
df.text = df.apply(lambda row: re.sub(r"http\S+", "", row.text).lower(), 1)df.text = df.apply(lambda row: " ".join(filter(lambda x:x[0]!="@", row.text.split())), 1)df.text = df.apply(lambda row: " ".join(re.sub("[^a-zA-Z]+", " ", row.text).split()), 1)
... or, if it’s already cleaned, we can prepare our two variables. The most critical variable is obviously our list of documents, which we will call titles. Secondly, we will want a list of dates corresponding to each document so that we can gain insights into how our topics have shifted with respect to time.
titles = df.text.to_list()dates = df['date'].apply(lambda x: pd.Timestamp(x)).to_list()
Next, let’s create our topic model!
from bertopic import BERTopictopic_model = BERTopic(min_topic_size=70, n_gram_range=(1,3), verbose=True)topics, _ = topic_model.fit_transform(titles)
We can extract the largest ten topics based on the number of topics assigned to each topic and also preview the keyword-based “names” of our topics, as alluded to earlier:
freq = topic_model.get_topic_info()freq.head(10)
Note that in the above data frame, topic -1 denotes the topic consisting of outlier documents which are typically ignored due to terms having a relatively high prevalence across the whole corpus and thus, low specificity toward any cohesive theme or topic.
We can also take a look at the terms which comprise a particular topic of interest:
topic_nr = freq.iloc[6]["Topic"] # select a frequent topictopic_model.get_topic(topic_nr)
Great! All of our least favorite words belong to Topic #5. We can see that the top three words are “coronavirus”, “covid”, and “pandemic”, but there is also an n-gram which has made it into these top 10 topic-specific terms: “covid vaccine,” and it’s a term which has more positive connotations, so it will be interesting to see at which point in time the word “vaccine” started appearing in article titles with greater frequency.
The cherry on top of this entire technique is the ability to visualize our topic model in a way that tells us enough about our data without needing to investigate the raw text itself.
If you recall from my previous topic modeling article entitled “NLP Preprocessing and Latent Dirichlet Allocation (LDA) Topic Modeling with Gensim,” there exists a Python visualization package called pyLDAvis, which enables the user to produce interactive subplots depicting the distance between topics on a 2D plane as well as the top 30 most relevant and salient terms within the topic. BERTopic has its own intertopic distance map implementation which includes a hover tooltip which reveals the number of documents designated to a particular topic and the top 5 most frequent terms within that topic. The line of code you will need to produce the following visualization is shockingly straightforward:
topic_model.visualize_topics()
The intertopic distance map above depicsts five main clusters of topics among our 15 total topics. The cluster that is closest to the x-axis pertains to the general theme of Python, which makes intuitive sense based on my Medium interests and search history. The cluster in the bottom left aligns consists of domain interests such as machine learning, deep learning, data science, tech design, etc. Gravitating toward the top left corner are articles that pertain to NLP, sentiment analysis, and Twitter data. Despite the fact that these could be labeled yet another domain interset of mine, I hypothesize that they occupy a different region of the 2D plane because I have researched these subjects with greater depth and granularity as part of my senior capstone project. The cluster in the middle bottom area near the y-axis, I presume, is characterized by less technical content, with subject lines that follow something along the lines of the templates “10 Tips to Avoid Burnout as a Software Engineer” or “What I learned from writing 100 Medium Articles in 100 Days”. These are the articles that lie at the intersection of productivity hacks and self-development wisdom, in my opinion. They don’t always teach me new things, but they certainly inspire me when I go through my Medium Digest each morning. Lastly, the rightmost topic cluster represents a potpourri of topics that do not reside within the tech space but are nevertheless important in order to consume a well-rounded selection of articles on Medium.
And now, for the main attraction, dynamic topic modeling with respect to time! In the video below, we can observe the frequency of the ten most prevalent topics in my corpus of suggested Medium articles (titles), from the start of 2020 through now.
We can observe that data science (topic 5) and development / coding (topic 9) have been consistent interests of mine over the course of almost 2 years. Interestingly, React apps (topic 6) were something that I read about extensively around October and November of 2020, but this interest of mine faded shortly thereafter. The backstory for this, you may be wondering? Well, I was cramming for a frontend development interview that I felt wildly unprepared for. Fun fact: Medium taught me pretty much everything I know about React. We can notice that the NLP topic (topic 8) spiked in frequency around the summer of 2020 and June of 2021, which makes a lot of sense when I reflect on what I was working on at those times. COVID-19 (topic 2) spiked in frequency around March through May of 2020, which speaks for itself I think... moving on. The potpourri topic (topic 0) had the highest frequency in early 2020, but tapered off later in the year. I guess I read a lot more non-technical content when I was stuck at home, whereas I now primarily consult Medium articles for technical purposes.
Without going into more personal reflection, I hope you get the gist of how powerful dynamic topic modeling can by. Hovering over our topic frequency line plots at any point in time brings up different topic keywords, thus making it possible to analyze how a topic has evolved in composition over time.
Though I briefly showed how to access the top keywords belonging to a particular topic and their importance scores, we can also visualize these terms and scores as bar charts.
That’s a wrap for our comprehensive tour of BERTopic! I hope you learned a thing or two about dynamic topic modeling and can see why I find this topic modeling Python package to surpass other packages and techniques in this space. Again, huge shoutout to the author of BERTopic, Maarten Grootendorst for implementing and open sourcing this awesome code.
If you want to give dynamic topic modeling a whirl, I have gone ahead and written a small Streamlit app which allows you to import a CSV file consisting of dates and documents from which you’d like to build a topic model. It then creates a BERTopic model and embeds some interactive visualizations within the app. I hope you enjoy and can play around with the app to your heart’s content. My motivation for developing this app was the following:
make topic modeling more interactive and enjoyable and move away from it being an intimidatingly technical NLP subdomainmake this subject matter accessible to less technical folks who may also want the opportunity to better understand the themes present in a given corpus of text
make topic modeling more interactive and enjoyable and move away from it being an intimidatingly technical NLP subdomain
make this subject matter accessible to less technical folks who may also want the opportunity to better understand the themes present in a given corpus of text
The app is accessible here: https://share.streamlit.io/sejaldua/digesting-the-digest/main/bertopic_app.py
As always, I am making my source code available in a public GitHub repository: