Sentiment Analysis: Aspect-Based Opinion Mining | by Lowri Williams | Towards Data Science

When growing a successful business, one of the most crucial elements is the communication and the relationships you have with your customers. However, one of the main challenges with a growing user base is the large amount of data generated, often in the form of natural language. This data can be driven from customer feedback or even on social media platforms where customers express their thoughts and opinion towards their experiences using your service or products.
The real challenge is to automatically parse and organise this data into a more digestible and actionable insight. One of the most popular natural language processing techniques to apply on such data is Sentiment Analysis. Sentiment Analysis aims to automatically extract and classify sentiments (the subjective part of an opinion) and/or emotions (the projections or display of a feeling) expressed in text.
An opinion may be defined as a combination of four factors (entity, holder, claim, and sentiment), in which the opinion holder may believe a claim about an entity, and in many cases, associate a sentiment with that belief. As opposed to extracting the general sentiment expressed in a piece of text, Aspect-Based Sentiment Analysis aims to extract both the entity described in the text (in this case, attributes or components of a product or service) and the sentiment expressed towards such entities.
The advantages gained from exploring how your customers are reacting towards particular parts of your service or product can help support business use cases, including product development and quality control, communications, customer support, and decision making processes. In this case, this post investigates combining traditional topic modelling techniques with sentiment analysis to extract text aspects, as well as the sentiment expressed towards them.
The dataset used in this post is by Julian McAuley. It contains product reviews and metadata from Amazon, including 142.8 million reviews spanning between May 1996 — July 2014. This dataset includes reviews (ratings, text, helpfulness votes), product metadata (descriptions, category information, price, brand, and image features), and links (also viewed/also bought graphs). Here, we’ll use reviews for cell phones and accessories. The total number of reviews in this dataset is 194,439.
We’ll follow the standard set of pre-processing techniques used in several natural language processing tasks. Such techniques include:
Converting the text to lowercase.
Removing punctuation and additional white spaces.
Tokenisation.
Removing stop words.
Lemmatisation.
Before we look into applying topic modelling techniques, the last pre-processing step is to vectorise the reviews, i.e. we need to represent the data into a numerical form so that the model can handle them. There are several representations you can use, with the popular methods being the word’s TF-IDF score or their frequency counts (bag-of-words approach). Here, we’ll stick to a bag-of-words representation. We’ll use the CounterVectorizer function from Sklearn’s feature extraction module. This function converts a collection of text to a matrix of word counts.
# initialise the count vectorizer

vectorizer = CountVectorizer(analyzer = 'word', ngram_range = (1, 2))
                            
# join the processed data to be vectorised

vectors = []

for index, row in df.iterrows():
    vectors.append(", ".join(row[1]))

vectorised = vectorizer.fit_transform(vectors)

print(vectorised)

  (0, 8018)	1
  (0, 4092)	1
  (0, 9224)	1
  (0, 3321)	1
  (0, 13901)	1
  (0, 5656)	1
  (0, 8020)	1
  (0, 4112)	1
  (0, 9227)	1
  (0, 3329)	1
  (0, 13903)	1
  (1, 7361)	1
  (1, 11595)	1
  (1, 12200)	1
  (1, 6240)	1
  (1, 10331)	1
  (1, 7062)	1
  (1, 7380)	1
  (1, 11621)	1
  (1, 12201)	1
  (1, 6247)	1
  (1, 10333)	1
  (2, 5582)	1
  (2, 13911)	1
  (2, 13909)	1
  :	:
  (2504, 4458)	1
  (2504, 16160)	1
  (2504, 4108)	1
  (2504, 6052)	1
  (2504, 6053)	1
  (2504, 8196)	1
  (2504, 4555)	1
  (2505, 6660)	1
  (2505, 2608)	1
  (2505, 16288)	1
  (2505, 13932)	1
  (2505, 10991)	1
  (2505, 1308)	1
  (2505, 2609)	1
  (2505, 10853)	1
  (2505, 16293)	1
  (2505, 10855)	1
  (2505, 13935)	1
  (2505, 10992)	1
  (2505, 6664)	1
  (2506, 12878)	1
  (2506, 16264)	1
  (2506, 4699)	1
  (2506, 16267)	1
  (2506, 12880)	1

Topic modelling is an unsupervised machine learning approach used to distribute texts into groups which best characterise such documents. This concept can also be used to extract text aspects.
Sklearn includes a version of the Latent Dirichlet Allocation (LDA) algorithm. For this concept, we want to extract five aspects. Once the parameters are set, we can fit the LDA to the vectorised version of the text.
To make the output easier to read, we can append the relevance score produced for each topic to each review as a column and calculate the dominant topic by taking the topic with the highest relevance score.
# initisalise LDA Model

lda_model = LatentDirichletAllocation(n_components = 5, # number of topics
                                  random_state = 10,          # random state
                                  evaluate_every = -1,      # compute perplexity every n iters, default: Don't
                                  n_jobs = -1,              # Use all available CPUs
                                 )

lda_output = lda_model.fit_transform(vectorised)

# column names

topic_names = ["Topic" + str(i) for i in range(1, lda_model.n_components + 1)]

# make the pandas dataframe

df_document_topic = pd.DataFrame(np.round(lda_output, 2), columns = topic_names)

# get dominant topic for each document

dominant_topic = (np.argmax(df_document_topic.values, axis=1)+1)
df_document_topic['Dominant_topic'] = dominant_topic

# join to original dataframes

df = pd.merge(df, df_document_topic, left_index = True, right_index = True, how = 'outer')
display(df.head(10))

Now we can see which topic (by its number) a review belongs to. But what keywords, in this case text aspects, has the LDA extracted from such reviews? We can view them by calling the vectoriser’s get_feature_names() and see their relevance score to that aspect using the LDA’s components_functions:
# index names
docnames = ['Doc' + str(i) for i in range(len(data))]

# Make the pandas dataframe
df_document_topic = pd.DataFrame(np.round(lda_output, 2), columns=topic_names, index=docnames)

# Get dominant topic for each document
dominant_topic = np.argmax(df_document_topic.values, axis=1)
df_document_topic['dominant_topic'] = dominant_topic

# Topic-Keyword Matrix
df_topic_keywords = pd.DataFrame(lda_model.components_)

# Assign Column and Index
df_topic_keywords.columns = vectorizer.get_feature_names()
df_topic_keywords.index = topicnames

df_topic_no = pd.DataFrame(df_topic_keywords.idxmax())
df_scores = pd.DataFrame(df_topic_keywords.max())

tmp = pd.merge(df_topic_no, df_scores, left_index=True, right_index=True)
tmp.columns = ['topic', 'relevance_score']

display(tmp)

1358778 rows × 2 columns
We determine which aspect a keyword belongs to by taking its highest relevance score across the five aspects. Then, for each aspect, we can order our dataframe in descending order and select the keywords with the highest score.
The five aspects are:
Battery charger
Battery pack
Car charger
Cell phone
Screen protector
all_topics = []

for i in tmp['topic'].unique():    
    tmp_1 = tmp.loc[tmp['topic'] == i].reset_index()
    tmp_1 = tmp_1.sort_values('relevance_score', ascending=False).head(1)

    tmp_1['topic'] = tmp_1['topic'] + 1
    
    tmp_2 = []
    tmp_2.append(tmp_1['topic'].unique()[0])
    tmp_2.append(list(tmp_1['index'].unique()))
    all_topics.append(tmp_2)

all_topics = pd.DataFrame(all_topics, columns=['Dominant_topic', 'topic_name'])
display(all_topics)

Various techniques and methodologies have been developed to address automatically identifying the sentiment expressed in the text. In this post, I’ll use VADER, a Python sentiment analysis library, to classify whether the reviews are positive, negative, or neutral.
A very simple approach to sentiment analysis is by using a list of words which have been labelled according to their semantic orientation. For example, we can assume that the word “good” has a positive valence, whereas the word “bad” has a negative one. VADER uses this technique and provides a percentage score which represents the proportion of lyrics which fall in each sentiment category. It also provides a compound score which is computed by summing the valence scores of each word and then normalising the scores to be between -1 (most extreme negative) and +1 (most extreme positive).
What’s nice about VADER is the fact that we don’t have to pre-process the text in any way. We can feed the raw product reviews into VADER’s sentiment function and retrieve the compound scores for each.
First, we’ll import the SentimentIntensityAnalyzer function from VADER’s Python library. I’ll initialise the sentiment analyser from VADER, and then iterate over the reviews from the dataframe. I’ll then calculate whether the compound sentiment score is above or beneath the thresholds so that we can assign them with the positive, negative, or neutral label.
Once we’ve retrieved the compound scores for each review, we can plot and count the number of positive, negative, and neutral reviews for each of the five aspects.
This chart reports that the majority of the reviews for each aspect are positive. In particular, with more than 83,000 reviews, the screen protector received 49,572 positive ones. For a business, having more than 50% of your reviews expressing a positive sentiment is a good indication that your customers are satisfied with the product.
What other potential insights can we gather? Let’s dive deeper into what the customers are saying. For this concept, we’ll use the reviews for the screen protector.
One of the main features used to support Sentiment Analysis include individual sentiment words (e.g. good, bad) which explicitly convey a subjective bias. Sentiment words are available from specialised dictionaries, where they have been mapped to their sentiment.
These words are often adjectives (e.g. good, bad), adverbs (e.g. cheerfully, weirdly), nouns (e.g. blessing, rubbish), and verbs (e.g. love, hate). Sentiment may also be expressed by using comparative words (e.g. better, worse). To identify those adjectives, adverbs, nouns, and verbs in text, we can apply Part of Speech (POS) tagging. POS tagging is the process of marking up a word in a text as corresponding to a particular part of speech (noun, verb, adjective, adverb, pronoun, preposition, conjunction, interjection, numeral, article, or determiner), based on both its definition and its context.
Here, we use the NLTK POS tagger to identify the POS tags. As we’ve got quite a large dataset, we want to automatically narrow down the sentiment words as much as possible. In this case, we’re able to identify English and correctly spelt words and map them to SentiWordNet, a lexical resource for opinion mining. SentiWordNet assigns words three sentiment scores: positivity, negativity, and objectivity.
Objectivity can be defined as not being influenced by personal feelings or opinions in considering and representing facts. Subjectivity is therefore the contrast of objectivity. In terms of scoring, we want to identify words with low objectivity score. But not all subjective words are sentiment words.
To capture sentiment words from subjective ones, we use WordNet-Affect, another lexical resource which represents affective concepts correlated with affective words in a hierarchical structure. Once we’ve checked whether the subjective words are in WordNet-Affect, we can be quite confident that these are sentiment words. Based on their SentiWordNet positive and negative score, we can also split them up into separate collections and count how many times they appear across the whole dataset.
We’ll illustrate the results in a word cloud, a simple visualisation of data in which words are shown in varying sizes depending on how often they appear in your data. The size of a word shows how often it appears in a review.
Based on both word clouds, it’s quite clear that the most common positive and negative sentiment words used in the reviews are happy and worry respectively. For a business, again, having happy as the most popular positive sentiment word is reassuring!
“I love this case! It’s so pretty. And I love the way the case feels to the touch because of the rubber. Very happy!”
But the negative sentiment words might be concerning. After scanning over the reviews which contain the word worry, we can note that the majority of their occurrences are after the negations ‘didn’t worry’ or ‘don’t worry’. Negation plays an important role in sentiment analysis as it can revert the polarity of sentiment words. Negation words such as ‘not’ or ‘never’ are often considered stop words, which were removed during pre-processing and their effect on the polarity of sentiment words. This is an important future note to consider.
The following review illustrates an the example where hate, the second most popular negative sentiment word across the reviews, is present:
“I hate screen protectors. HATE them, and thus rarely use them. They’re either a pain in the ass to install, make the display look like crap, or a combination of both.”
Guess this person dislikes screen protectors...
So, what have I learnt from this analysis?
Traditional topic modelling techniques can be used to successfully extract aspects from text. Like many data science problems, one of the core tasks of the problem is the pre-processing of the data. This has particular effects on sentiment analysis results, where removing specific stop words may impact the identification of negation and therefore the true sentiment expressed.
For the full notebook, check out my GitHub repo below:
https://github.com/LowriWilliams/Aspect_Sentiment_Analysis
If you’re interested in more topic modelling posts, I recently wrote about determining more meaningful titles to a given topic. You can read it here:
🗣️ Topic Modelling: Going Beyond Token Outputs