Package: btmodified
BERTopicBERTopic is a topic modeling technique that leverages ðŸ¤— transformers and c-TF-IDF to create dense clusters
allowing for easily interpretable topics whilst keeping important words in the topic descriptions. It even supports
visualizations similar to LDAvis!Corresponding medium post can be foundhereandhere.InstallationInstallation can be done usingpypi:pipinstallbertopicTo use the visualization options, install BERTopic as follows:pipinstallbertopic[visualization]To use Flair embeddings, install BERTopic as follows:pipinstallbertopic[flair]Getting StartedFor an in-depth overview of the features ofBERTopicyou can check the full documentationhereor you can follow along
with the Google Colab notebookhere.Quick StartWe start by extracting topics from the well-known 20 newsgroups dataset which is comprised of english documents:frombertopicimportBERTopicfromsklearn.datasetsimportfetch_20newsgroupsdocs=fetch_20newsgroups(subset='all',remove=('headers','footers','quotes'))['data']topic_model=BERTopic()topics,_=topic_model.fit_transform(docs)After generating topics and their probabilities, we can access the frequent topics that were generated:>>>topic_model.get_topic_freq().head()TopicCount-17288493992307012768411568-1 refers to all outliers and should typically be ignored. Next, let's take a look at the most
frequent topic that was generated,topic 49:>>>topic_model.get_topic(49)[('windows',0.006152228076250982),('drive',0.004982897610645755),('dos',0.004845038866360651),('file',0.004140142872194834),('disk',0.004131678774810884),('mac',0.003624848635985097),('memory',0.0034840976976789903),('software',0.0034415334250699077),('email',0.0034239554442333257),('pc',0.003047105930670237)]NOTE: UseBERTopic(language="multilingual")to select a model that supports 50+ languages.Visualize TopicsAfter having trained ourBERTopicmodel, we can iteratively go through perhaps a hundred topic to get a good
understanding of the topics that were extract. However, that takes quite some time and lacks a global representation.
Instead, we can visualize the topics that were generated in a way very similar toLDAvis:topic_model.visualize_topics()Embedding ModelsThe parameterembedding_modeltakes in a string pointing to a sentence-transformers model,
a SentenceTransformer, or a Flair DocumentEmbedding model.Sentence-TransformersYou can select any model fromsentence-transformershereand pass it through BERTopic withembedding_model:frombertopicimportBERTopictopic_model=BERTopic(embedding_model="xlm-r-bert-base-nli-stsb-mean-tokens")Or select a SentenceTransformer model with your own parameters:frombertopicimportBERTopicfromsentence_transformersimportSentenceTransformersentence_model=SentenceTransformer("distilbert-base-nli-mean-tokens",device="cpu")topic_model=BERTopic(embedding_model=sentence_model)FlairFlairallows you to choose almost any embedding model that
is publicly available. Flair can be used as follows:frombertopicimportBERTopicfromflair.embeddingsimportTransformerDocumentEmbeddingsroberta=TransformerDocumentEmbeddings('roberta-base')topic_model=BERTopic(embedding_model=roberta)You can select any ðŸ¤— transformers modelhere.Custom EmbeddingsYou can also use previously generated embeddings by passing it throughfit_transform():topic_model=BERTopic()topics,_=topic_model.fit_transform(docs,embeddings)Dynamic Topic ModelingDynamic topic modeling (DTM) is a collection of techniques aimed at analyzing the evolution of topics
over time. These methods allow you to understand how a topic is represented across different times.
Here, we will be using all of Donald Trump's tweet so see how he talked over certain topics over time:importreimportpandasaspdtrump=pd.read_csv('https://drive.google.com/uc?export=download&id=1xRKHaP-QwACMydlDnyFPEaFdtskJuBa6')trump.text=trump.apply(lambdarow:re.sub(r"http\S+","",row.text).lower(),1)trump.text=trump.apply(lambdarow:" ".join(filter(lambdax:x[0]!="@",row.text.split())),1)trump.text=trump.apply(lambdarow:" ".join(re.sub("[^a-zA-Z]+"," ",row.text).split()),1)trump=trump.loc[(trump.isRetweet=="f")&(trump.text!=""),:]timestamps=trump.date.to_list()tweets=trump.text.to_list()Then, we need to extract the global topic representations by simply creating and training a BERTopic model:frombertopicimportBERTopicmodel=BERTopic(verbose=True)topics,_=model.fit_transform(tweets)From these topics, we are going to generate the topic representations at each timestamp for each topic. We do this
by simply callingtopics_over_timeand pass in his tweets, the corresponding timestamps, and the related topics:topics_over_time=model.topics_over_time(tweets,topics,timestamps)Finally, we can visualize the topics by simply callingvisualize_topics_over_time():model.visualize_topics_over_time(topics_over_time,top_n=6)OverviewMethodsCodeFit the modeltopic_model.fit(docs])Fit the model and predict documentstopic_model.fit_transform(docs])Predict new documentstopic_model.transform([new_doc])Access single topictopic_model.get_topic(12)Access all topicstopic_model.get_topics()Get topic freqtopic_model.get_topic_freq()Get all topic informationtopic_model.get_topic_info()Visualize Topicstopic_model.visualize_topics()Visualize Topic Probability Distributiontopic_model.visualize_distribution(probabilities[0])Update topic representationtopic_model.update_topics(docs, topics, n_gram_range=(1, 3))Reduce nr of topicstopic_model.reduce_topics(docs, topics, nr_topics=30)Find topicstopic_model.find_topics("vehicle")Save modeltopic_model.save("my_model")Load modelBERTopic.load("my_model")Get parameterstopic_model.get_params()CitationTo cite BERTopic in your work, please use the following bibtex reference:@misc{grootendorst2020bertopic,author={Maarten Grootendorst},title={BERTopic: Leveraging BERT and c-TF-IDF to create easily interpretable topics.},year=2020,publisher={Zenodo},version={v0.5.0},doi={10.5281/zenodo.4430182},url={https://doi.org/10.5281/zenodo.4430182}}