Topic Modeling with NLP on Amazon Reviews | by Enes Gokce | Towards Data Science

Topic modeling is another popular text analysis technique. The ultimate goal of topic modeling to find a theme across reviews, and discover hidden topics. Each document in the corpus will be made up of at least one topic, if not multiple topics.
In this part, I will be covering the results of Latent Dirichlet Allocation (LDA), which is one of many topic modeling techniques. It’s an unsupervised classification method. When we have a large text and don’t know where to start, then we can apply topic modeling and can see what kind of groups are emerging. LDA was specifically designed for text data.
To use a topic modeling technique, you need to provide:
A document-term matrixThe number of topics you would like the algorithm to pick up.
A document-term matrix
The number of topics you would like the algorithm to pick up.
In topic modeling, I will create different models and compare them. In the end, I will choose the topic model that makes the most sense.
In here, I won’t cover the mathematical foundation of the LDA. I will just discuss how to interpret the results of LDA topic modeling.
During LDA Topic modeling, we create many different topic groups. As a researcher, we are the one who decides the number of groups in the output. However, we do not know what is the best number of groups. Therefore, we will obtain different numbers of groups. Then, we examine and compare topic modelings, and decide which topic model makes more sense, most meaningful, have the clearest distinction within the model. Then, the group (model) that makes the most sense will be chosen among all topic groups.
It must be noted that the nature of the LDA is subjective. It’s possible that different people may reach different conclusions about choosing the most meaningful topic groups. We are looking for the most reasonable topic groups. People from different backgrounds, different domain expertise may not be on the same page about most sensible topic groups.
LDA is an unsupervised clustering method. When we are talking about unsupervised clustering methods, we need to mention K-Means Clustering. K-Means Clustering is one of the most well-known unsupervised clustering methods. It’s very practical and useful in many cases and has used for text mining for years. In contrast to K-means clustering, where each word can only belong to one cluster (hard-clustering), LDA allows for ‘fuzzy’ memberships (soft clustering). Soft-clustering allows overlap among the cluster, whereas, in hard-clustering, clusters are mutually exclusive. What does this mean? In LDA, a word can belong to more than one group while this is not possible in K-Means clustering. In LDA, this trade-off makes it easier to find similarities between the words. However, this soft-clustering makes it hard to obtain distinct groups because the same words can appear in different groups. We will experience this drawback in our further analysis.
Once the topic modeling technique is applied, the researcher’s job as a human is to interpret the results and see if the mix of words in each topic makes sense. If they don’t make sense, we can try changing up the number of topics, the terms in the document-term matrix, model parameters, or even try a different model.
Brief information about the data I use: The data used in this study were downloaded from Kaggle. It was uploaded by the Stanford Network Analysis Project. The original data is coming from the study of ‘From amateurs to connoisseurs: modeling the evolution of user expertise through online reviews’ done by J. McAuley and J. Leskovec (2013). This data set consists of reviews of fine foods from Amazon. The data includes all 568,454 reviews spanning 1999 to 2012. Reviews include product and user information, ratings, and a plain text review.
In this study, I will focus on ‘good reviews’ on Amazon. I defined ‘good reviews’ as the reviews that have 4-star or 5-star customer reviews (out of 5). In other words, if an Amazon review is 4 or 5 stars, it’s called ‘good review’ in this study. 1,2 or 3-star reviews are labeled as ‘bad reviews’.
Data preparation is a critical part because if we fail to prepare the data properly, we can’t perform topic modeling. Here, I won’t dive into detail because data preparation is not the focus of this study. However, be ready to spend some time here in case you have a problem. If you adjust the codes I provide here for your own dataset, I hope you should not have any problem.
Let’s start with checking the number of column and row of the data:
df.shape
(392384, 19) — my dataset has 392,384 reviews.
This amount of review is so hard to handle for many home computers (and for the Google Colab). Therefore, I will use just 10,000 reviews. Unfortunately, we can’t use all of our reviews if we don’t have access to a supercomputer.
#Taking only first 10000 Good Reviews from the dataset.df_good_reviews= df.loc[df.Good_reviews ==1][0:10000]
Now, the next step of data preparation is the count vectorizer:
# Create document-term matrixfrom sklearn.feature_extraction import textfrom sklearn.feature_extraction.text import CountVectorizercv = CountVectorizer()data_cv = cv.fit_transform(df_good_reviews.Text)data_cv.shape
Pickling the data for creating a term-document matrix
# Pickle it import picklepickle.dump(cv, open("cv_stop.pkl", "wb"))data_stop.to_pickle("dtm_stop.pkl") #saving the pickled datadata = pd.read_pickle('dtm_stop.pkl')data
After pickling the data, we obtain a wide word dictionary:
Now, it’s time to create the term-document matrix. This term-document matrix data will be used for generating topic modeling. That’s why it’s crucial for the study.
# One of the required inputs is a term-document matrixtdm = data.transpose()tdm.head()
# We're going to put the term-document matrix into a new gensim format, from df --> sparse matrix --> gensim corpussparse_counts = scipy.sparse.csr_matrix(tdm)corpus = matutils.Sparse2Corpus(sparse_counts)# Gensim also requires dictionary of the all terms and their respective location in the term-document matrixcv = pickle.load(open("cv_stop.pkl", "rb"))id2word = dict((v, k) for k, v in cv.vocabulary_.items())
Let’s start to create topic modelings!
There are many ways to create topic modelings. By filtering the text for certain criteria, we can obtain different topic groups. In this study, we will create topic modeling by using:
All the text dataOnly nouns from the textOnly nouns and adjectives
All the text data
Only nouns from the text
Only nouns and adjectives
Topic Modeling — Attempt 1 (with all the review data)
As a beginning, we are using all the reviews that we have. No text filtering is applied in this process.
lda = models.LdaModel(corpus=corpus, id2word=id2word, num_topics=2, passes=10)lda.print_topics()
Discovered two groups of topics:
[(0, ‘0.020*”coffee” + 0.011*”like” + 0.010*”tea” + 0.010*”flavor” + 0.009*”good” + 0.009*”taste” + 0.007*”one” + 0.007*”great” + 0.006*”use” + 0.006*”cup”’),
(1, ‘0.010*”great” + 0.009*”like” + 0.009*”good” + 0.007*”love” + 0.007*”food” + 0.007*”one” + 0.007*”product” + 0.005*”taste” + 0.005*”get” + 0.005*”amazon”’)]
What will I do with this result? At this point, we can examine these two topic groups and check whether they are distinct groups or not. We don’t expect all words in a group must be related. We are looking for a general trend in the topic group. Considering the above two groups, it’s hard to see a clear distinct group there. This is my opinion. If you think that there are actually two discrete groups, and you can justify it, then you are safe to go with it.
Remember, I said that LDA is a soft-clustering model. Here, we see that the word ‘like’ appeared in both groups. This is normal because, in soft-clustering models, a word can appear in different groups at the same time.
We don’t need to push ourselves to give meaning to the above model. We can move forward. Let’s create more topic models.
Similar to the 2 topics model, we will create 3 and 4 groups.
# LDA for num_topics = 3lda = models.LdaModel(corpus=corpus, id2word=id2word, num_topics=3, passes=10)lda.print_topics()# LDA for num_topics = 4lda = models.LdaModel(corpus=corpus, id2word=id2word, num_topics=4, passes=10)lda.print_topics()
All the results can be seen in Table 1 (below table). At this point, we should examine the groups and try to find a group that makes the most sense. By looking at Table 1, it can be said that ‘Modeling with 2 Topics’ makes the most sense. The first group is about beverages, and the second group is about reactions. Feel free to reach a different conclusion than me! For now, this will stay here and won’t go further because we will create 6 more topic models. I can check the outputs more closely after I get all results. We will reconsider Table 1 after producing further topic modelings.
Topic Modeling — Attempt 2 (Nouns only): In this step, only nouns will be used for creating topics by using the LDA method. Again, our purpose is to find hidden patterns in reviews. Now, we are just trying to explore by using different criteria.
Similar to the previous step, we will run LDA with 2,3 and 4 topic groups.
import nltkfrom nltk.corpus import stopwordsnltk.download('punkt')nltk.download('averaged_perceptron_tagger')import stringfrom nltk import word_tokenize, pos_tag
Let's create a function to pull out nouns from a string of text:
from nltk import word_tokenize, pos_tagdef nouns(text):    '''Given a string of text, tokenize the text and pull out only the nouns.'''    is_noun = lambda pos: pos[:2] == 'NN'    tokenized = word_tokenize(text)    all_nouns = [word for (word, pos) in pos_tag(tokenized) if     is_noun(pos)]    return ' '.join(all_nouns)#if this code doesn't work on you, it's probably indentaion mistake coming from Medium writing format.
Let's apply this defined noun function to our Amazon Review text data:
data_nouns = pd.DataFrame(df_good_reviews.Text.apply(nouns))data_nouns
Create a new document-term matrix using only nouns:
from sklearn.feature_extraction import textfrom sklearn.feature_extraction.text import CountVectorizer# Re-add the additional stop words since we are recreating the document-term matrixadd_stop_words = ['like', 'im', 'know', 'just', 'dont', 'thats', 'right', 'people','youre', 'got', 'gonna', 'time', 'think', 'yeah', 'said']stop_words = text.ENGLISH_STOP_WORDS.union(add_stop_words)# Recreate a document-term matrix with only nounscvn = CountVectorizer(stop_words=stop_words)data_cvn = cvn.fit_transform(data_nouns.Text)data_dtmn = pd.DataFrame(data_cvn.toarray(), columns=cvn.get_feature_names())data_dtmn.index = data_nouns.indexdata_dtmn
Create the gensim corpus:
corpusn=matutils.Sparse2Corpus(scipy.sparse.csr_matrix(data_dtmn.transpose()))# Create the vocabulary dictionaryid2wordn = dict((v, k) for k, v in cvn.vocabulary_.items())
⚫ Let’s start with 2 topics
ldan = models.LdaModel(corpus=corpusn, num_topics=2, id2word=id2wordn, passes=10)ldan.print_topics()
⚫ LDA with 3topics
ldan = models.LdaModel(corpus=corpusn, num_topics=3, id2word=id2wordn, passes=10)ldan.print_topics()
⚫ LDA with 4 topics
ldan = models.LdaModel(corpus=corpusn, num_topics=4, id2word=id2wordn, passes=10)ldan.print_topics()
Table 2 is the output of the LDA topic modeling attempts with only nouns. At this point, again we need to briefly examine topics and try to find a model that has distinct topic groups. Again, still no need to spend much time because we will still produce more topic models.
To me, The topic model with three groups is meaningful. It has these groups:
Pet itemsCookies and snacksDrinkings
Pet items
Cookies and snacks
Drinkings
It’s totally normal that people can reach different conclusions than others.
Topic Modeling — Attempt 3 (Nouns and Adjectives): In this step, only nouns and adjectives were used for creating topics by using the LDA method.
Let’s prepare the data for this:
from nltk.corpus import stopwordsimport nltknltk.download('punkt')nltk.download('averaged_perceptron_tagger')import stringfrom nltk import word_tokenize, pos_tag
Let’s define the function that will give us only nouns or adjectives:
def nouns_adj(text):    '''Given a string of text, tokenize the text and pull out only     the nouns and adjectives.'''    is_noun_adj = lambda pos: pos[:2] == 'NN' or pos[:2] == 'JJ'    tokenized = word_tokenize(text)    nouns_adj = [word for (word, pos) in pos_tag(tokenized) if   is_noun_adj(pos)]    return ' '.join(nouns_adj)#if this code doesn't work on you, it's probably indentaion mistake coming from Medium writing format.
Let’s check the Amazon Review data with noun/adj only filter:
data_nouns_adj = pd.DataFrame(df_good_reviews.Text.apply(nouns_adj))data_nouns_adj
As we can see, we only have nouns and adjectives here. Now, we will ask LDA to create topics by using this filtered version of our dataset.
Let’s prepare the data for LDA:
# Create a new document-term matrix using only nouns and adjectives, also remove common words with max_dfcvna = CountVectorizer(max_df=.8) #, max_df is used for removing data values that appear too frequently, also known as "corpus-specific stop words".# For example, max_df=.8 means "It ignores terms that appear in more than 80% of the documents". data_cvna = cvna.fit_transform(data_nouns_adj.Text)data_dtmna = pd.DataFrame(data_cvna.toarray(), columns=cvna.get_feature_names())data_dtmna.index = data_nouns_adj.indexdata_dtmna
Create the gensim corpus:
corpusna=matutils.Sparse2Corpus(scipy.sparse.csr_matrix(data_dtmna.transpose()))# Create the vocabulary dictionaryid2wordna = dict((v, k) for k, v in cvna.vocabulary_.items())
We can run LDA now.
⚫ Let’s start with 2 topics
ldana = models.LdaModel(corpus=corpusna, num_topics=2, id2word=id2wordna, passes=10)ldana.print_topics()
⚫ Let’s try with 3 topics
ldana = models.LdaModel(corpus=corpusna, num_topics=3, id2word=id2wordna, passes=10)ldana.print_topics()
⚫ Let’s try with 4 topics
ldana = models.LdaModel(corpus=corpusna, num_topics=4, id2word=id2wordna, passes=10)ldana.print_topics()
Table 3 is the output of the LDA topic modeling attempts with only nouns or adjectives. At this point, again we need to examine topics and check if there is any topic group making sense.
Now, in the final stage, Table 1, Table 2, and Table 3 must be evaluated together. In total, we created 9 topic models. At this point, we need to ask ourselves self ‘which group makes more sense?’. Now, it’s time to focus and examine the topics. Which groups make sense? If there is no group that makes sense, we need to go back data to data cleaning, change parameters, or use different filters and models. It’s a recursive process.
Out of the 9 examined topic models, nouns only, the 3-topic group made the most sense to me (Table 4). I see three distinct groups here: (1) pet items, (2) cookies and snacks, and (3) beverage. Again, it’s totally normal that another person can find a different topic model more meaningful.
Keep in mind that this dataset consists of only food reviews. So, it quite normal to see that the groups are related to foods.
So, let’s pull that down here and run it through some more iterations to get more fine-tuned topics. 3 topic (only with noun) was the most meaningful model to me. So, I will fine-tune this topic model more by increasing number of iterations from 10 to 80.
# Fine-tuned LDA with topics = 3ldan = models.LdaModel(corpus=corpusn, num_topics=3, id2word=id2wordn, passes=80)ldan.print_topics()
In the above table, we see similar groups with Table 4, just in different orders. By considering all the steps of topic analysis with the LDA method, it can be concluded that good Amazon Reviews can be categorized into three main topics: (1) beverages, (2) pet items, and (3) cookies and snacks.
Lastly, we need to keep in mind that in order to understand which topic group makes more sense, we may need domain expertise. As a researcher, it’s our responsibility to justify our choice of topic group. As long as we can justify, we can decide any topic group as a final topic model.
Thank you for reading! Happy analysis!
*Special thanks to my friend Bibor Szabo for prividing valuable feedbacks and suggestions while writing this posting.
Note: For the Python codes for this study, you can check my GitHub folder. All the scripts used in this study are available in the Github folder. In addition, this topic modeling study is a part of the bigger project. You can check my previous posts for data cleaning and sentiment analysis if you are interested in to see previous steps.