Top2Vec: Internet News Topic Modeling
By: Kamil Polak
Tags: Topic Modeling, Lda, NLP, Machine Learning, Word2vec

Top2Vec: Internet News Topic Modeling

A simplified way for topic modeling in Python

Image from Pixabay

Topic modeling is used for discovering topics in a large collection of documents. The most widely used methods are Latent Dirichlet Allocation and Probabilistic Latent Semantic Analysis.

Despite their popularity, they have several weaknesses., e.g. they often require the number of topics to be known and texts preprocessing steps.

In this article, I will show you a simplified way for topic modeling using the Top2Vec algorithm.

What is Topic Modeling?

Topic modeling is an unsupervised machine learning technique that’s capable of scanning a set of documents, detecting word and phrase patterns within them, and automatically clustering word groups and similar expressions that best characterize a set of documents.

Since topic modeling doesn’t require training, it’s a quick and easy way to start analyzing your data. However, you can’t guarantee you’ll receive accurate results, which is why many businesses opt to invest time in training a topic classification model.

Topic models are also referred to as probabilistic topic models, which refers to statistical algorithms for discovering the latent semantic structures of an extensive text body.

In the age of information, the amount of the written material we encounter each day is simply beyond our processing capacity. Topic models can help to organize and offer insights for us to understand large collections of unstructured text bodies.

Originally developed as a text-mining tool, topic models have been used to detect instructive structures in data such as genetic information, images, and networks

What isTop2Vec?

Released in March 2020, Top2Vec is created by Dimo Angelov, with its paper published on ArXiv in August 2020. Despite being new, the algorithms used by Top2Vec are well-established — Doc2Vec, UMAP, HDBSCAN. Top2Vec also supports the use of embedding models like Universal Sentence Encoder and BERT.

Top2Vec is an algorithm for topic modeling and semantic search which automatically detects topics present in the text. Removing stop-words, lemmatization, stemming, and a priori knowledge of the number of topics are not required for Top2Vec to learn good topic vectors. This gives Top2Vec a major advantage over traditional methods.

Once you train the Top2Vec model you can:

Get number of detected topics.

Get topics.

Get topic sizes.

Get hierarchichal topics.

Search topics by keywords.

Search documents by topic.

Search documents by keywords.

Find similar words.

Find similar documents

How does Top2Vec work?

Top2Vec works based on assumption that many semantically similar documents are indicative of an underlying topic.

We can describe the the Top2Vec in three steps.

Create Semantic Embedding

Top2Vec creates numeric representations such that the semantic contents of the documents are captured, and similar documents and words will be close to each other

The algorithm is looking for an embedding where the distance between document vectors and word vectors represents the semantic association. It works based on assumption that semantically similar documents should be placed close together in the embedding space, and dissimilar documents should be placed further from each other.

2. Dimensionality reduction

Document vectors in high-dimensional space tend to be sparse. Top2Vec reduces dimensionality using UMAP (Uniform Manifold Approximation and Projection).

3. Clustering of documents to find topics

After dimensionality reduction algorithm is looking for dense areas of documents using HDBSCAN. HDBSCAN assigns a label to each dense cluster of document vectors and assigns a noise label to all document vectors that are not in a dense cluster. The dense areas of identified document vectors will be used to calculate the topic vectors.

The next step is to calculate, for each dense area, the topic vector. Finally, each document is assigned a topic number based on the nearest topic vector to its document vector.

Topic Modeling in Python with Top2Vec

Now I will show you practical implementation of Top2Vec in Python. I will use Kaggle Internet news data with readers engagement.

The internet news data was collected between 03.09.2019 and 04.11.2019 (DD.MM.YYYY). It contains articles (listed as the top in popularity at the publisher website) from multiple well-known publishers.

Import and Setup

1. Install the Top2Vec library

2. Import Libraries

3. Import dataset

Remove missing values

Train Top2Vec Model

Parameters:

documents : Input corpus, should be a list of strings.

: Input corpus, should be a list of strings. speed : This parameter will determine how fast the model takes to train. The ‘fast-learn’ option is the fastest and will generate the lowest quality vectors. The ‘learn’ option will learn better quality vectors but take a longer time to train. The ‘deep-learn’ option will learn the best quality vectors but will take significant time to train.

: This parameter will determine how fast the model takes to train. The ‘fast-learn’ option is the fastest and will generate the lowest quality vectors. The ‘learn’ option will learn better quality vectors but take a longer time to train. The ‘deep-learn’ option will learn the best quality vectors but will take significant time to train. workers: The amount of worker threads to be used in training the model. Larger amount will lead to faster training.

Explore Top2Vec Discovered Topics

1. Get Number of Topics

This will return the number of topics that Top2Vec has found in the data.

2. Get Topics

This will return the topics in decreasing size.

3. Generate Word Clouds

Using a topic number you can generate a word cloud. We will generate word clouds for topics 70 through 75.

4. Search Papers by Topic

We are going to search by topic 40

For each of the returned documents we are going to print its content, score and document number.

5. Similar Keywords

Search for similar words to football.

6. Search Papers by Keywords

Search documents for content semantically similar to games and players.

Conclusion

In this article, I showed you an alternative method for topic modeling which is the Top2Vec algorithm. Top2Vec has several advantages over traditional topic modeling methods like LDA, e.g. it does not require stop-word lists or lemmatization/stemming. Additionally, prior knowledge of the number of topics isnot required which makes it easier and faster to implement.

Furthermore , I explained how to implement it in Python. Specifically, we discussed how to:

get number of topics,

get topics,

generate word clouds,

search papers by topic and keyword,

find similar keywords.

Reference