Unsupervised Topic Modeling Using Natural Language Processing (NLP)
By: Crystal Huang
Tags: Metis, NLP, Covid 19, Unsupervised Learning, Naturallanguageprocessing

Backstory and Project Goal

Because of the rapid increase in scientific literature around COVID19, it is hard to keep up with the newest publications. Scientists and researchers are overwhelmed by the volume and struggle to find articles that are relevant to their work. In addition, the limitations of scientific conferences make it even harder to collaborate and stay up to date. However, it is crucial for scientists to be aware of ongoing research to find relevant publications.

So, the goal of this project is to build an unsupervised NLP model (Topic modeling and/or recommendation system) that helps researchers to navigate the current surge of papers about COVID-19, find what is relevant to their work, and uncover the hidden semantic relationships.

Data

The data used in this project came from the COVID-19 Open Research Dataset, which is a collection of over 500,000 scholarly articles about the novel coronavirus for use by the global research community.

I took the subset of articles from January 2020 to May 2021, which is about 260,000 articles, and used the abstract of the articles as text in this project.

Tools and Approaches Used

Python (pandas, numpy)

langdetect

regex, string

spaCy, scispaCy (“en_core_sci_lg” model for biomedical, scientific, and clinical vocabulary)

NLTK

Scikitlearn

Gensim — LDA

WordCloud

pyLDAvis

Streamlit

Heroku

Methodology

Text Preprocessing

Filtered out non-English articles with langdetect Used regex and string for simple cleaning Used ScispaCy’s (a spaCy package) “en_core_sci_lg” model for biomedical, scientific, and clinical vocabulary to do parts of speech tagging Used gensim for phrase detection Removed stopwords with NLTK, used Lemmatization, kept noun, adjective, verb, adverb, and Tokenization

Data Transformation

Dictionary — with gensim.corpora Corpus

Topic Modeling

I built my topic model using LDA from gensim since I want to discover latent relationships in the corpus.

I first tried 10 topics for my base model. But the topics are either too general or overlapping. While subjective inspection can be useful to evaluate a topic model, it was challenging and time-consuming for this large dataset. So I used coherence score to help find the optimal number of topics, which is 28 (coherence score: 0.523 vs baseline coherence score: 0.483).

Findings and Insights

Model Interpretation and Visualization

For interpretation, I used pyLDAvis to see the topic's similarity and relative frequency in the corpus.

A quick explanation of pyLDAvis — There are three important features of the pyLDAvis graph. First, each circle is a topic. The area of each circle is the topic prevalence. So The larger it is, the more articles are about that topic. Second, the distance between two circles is topic similarity. So The further they are, the more different they are. Lastly, on the right side, are the relevant terms listed which indicate the meaning of each topic. To learn more about the details of pyLDAvis, here’s a good resource.

At first glance of the pyLDAvis, there appears to be some overlapping of topics in my model. So I did some investigation on those overlapping groups. And I found that even though there are some overlapping, the topics are still distinguishable and meaningful. So overall, I’m satisfied with the final model performance.

Application Usage

With the LDA model, I assigned each article with a dominant topic and their relevance to the topic and grouped articles by topics for the recommendation system. So researchers can look up articles based on a topic that is related to their work.

I used Streamlit to develop the application and deployed the app on Heroku with a smaller dataset (due to the size limit on GitHub).

Here’s the app if you’d like to check it out!

Future Potential

Although the model is far from perfect, it showcases how topic modeling can be used to recommend articles in the same topic space.

I think definitely more work can be done on NLP preprocessing by tuning more hyperparameters. I would also like to see if how the research trends change over time.

It is always possible to extend the recommender — based on keywords, adding other filters like by author or time period, etc. Also, trying different models for the recommender, such as a higher-dimensional SVD-style model that may be able to make precise comparisons between different articles and match them.

References