Twitter US Airline Sentiment
Tags: Naturallanguageprocessing, Sentiment Analysis, Machine Learning, NLP, Classification Algorithms

Using Sentiment analysis to understand user feedback

Motivation:

The digital connectivity bestows immense power to the customers in terms of vocalizing their thoughts, opinions, and reviews on a brand. The customer views expressed on Twitter, Facebook, and other online forums are forming the base of customer strategy for brands worldwide.

Airlines can use Sentiment analysis or Opinion Mining of the posts shared on various social media platform to gain user insights and strategize their brands.

Problem :

In current data set (which can be found here on Kaggle), we have tweets for 6 US airlines.

We need to analyze how travelers in February 2015 expressed their feelings on Twitter i.e. whether the tweets are positive, negative or neutral.

Data Processing:

Initial Cleaning

After cleaning out non unique columns and columns with more than 90% missing values, we’ll have below columns:

tweet_id

airline_sentiment

airline_sentiment_confidence

negativereason

negativereason_confidence

airline

name

retweet_count

text

tweet_created

tweet_location

user_timezone

Let’s look at how our data set looks like:

dataset

Exploratory Data Analysis

Let’s look at distribution of positive, neutral and negative tweets across our dataset.

pie chart for positive, negative and neutral tweets.

Note, that more than 60% of the tweets are negative.

Below is the distribution of airlines across tweets:

pie chart for airlines in dataset

A trend on positive, neutral and negative tweets across airlines:

bar chart: positive, neutral and negative tweets vs airlines

From above graph we can see that

1. United, US Airways and American have substantially negative tweets, these also have got over all more tweets

2. Virgin America, Delta and Southwest have fairly balanced tweets

Now, let’s see the trend of number of negative tweets vs dates:

bar plot trend of number of negative tweets vs dates grouped by airlines

It should be noted that American started getting tweets from 22-Feb-2015 onwards.

Let’s look into negative reason’s marked by customers in their feedback:

bar plot trend of number of negative tweets vs dates grouped by airlines

As seen majority tweets have said the reason as

Customer service issue Late flight

Let us now, analyze the frequency of words appearing in negative tweets using WordCloud:

wordcloud for frequency of words appearing in negative tweets

Observe that flight, hour, help, time, hold, bag, plane are present more frequently in negative statements.

Similarly, let’s analyze the frequency of words appearing in positive tweets:

wordcloud for frequency of words appearing in positive tweets

Observe, that thank, flight, great, will, awesome and love are present more frequently in positive statements

Let us now start analyzing the tweets

We’ll now clean up the text data, for this we’ll follow the below steps:

Remove all the special characters convert all letters to lower case filter out English stop words stemmer / lemmetization (optional)

Next, we’ll vectorize tweet text using TF-IDF, which is a the measure to evaluate how important a word is to a document in a corpus.

vectorize using TF-IDF

We’ll now apply following models to predict sentiments from tweet text data:

LogisticRegression MultinomialNB DecisionTreeClassifier RandomForestClassifier KNeighborsClassifier

Accuracy score for each model:

LogisticRegression Accuracy Score : with accuar79.1%

precision recall f1-score support negative 0.93 0.81 0.87 3232

neutral 0.48 0.66 0.56 648

positive 0.60 0.81 0.69 512 accuracy 0.79 4392

macro avg 0.67 0.76 0.71 4392

weighted avg 0.83 0.79 0.80 4392MultinomialNB Accuracy Score : 69.69%

precision recall f1-score support negative 0.99 0.69 0.81 4081

neutral 0.15 0.78 0.26 174

positive 0.18 0.93 0.31 137 accuracy 0.70 4392

macro avg 0.44 0.80 0.46 4392

weighted avg 0.94 0.70 0.77 4392DecisionTreeClassifier Accuracy Score : 67.42%

precision recall f1-score support negative 0.79 0.78 0.79 2841

neutral 0.40 0.41 0.40 879

positive 0.55 0.57 0.56 672 accuracy 0.67 4392

macro avg 0.58 0.58 0.58 4392

weighted avg 0.68 0.67 0.67 4392RandomForestClassifier Accuracy Score : 76.78%

precision recall f1-score support negative 0.94 0.79 0.86 3378

neutral 0.38 0.63 0.48 535

positive 0.54 0.79 0.64 479 accuracy 0.77 4392

macro avg 0.62 0.74 0.66 4392

weighted avg 0.83 0.77 0.79 4392KNeighborsClassifier Accuracy Score : 69.83%

precision recall f1-score support negative 0.82 0.80 0.81 2866

neutral 0.46 0.42 0.44 960

positive 0.53 0.65 0.58 566 accuracy 0.70 4392

macro avg 0.60 0.62 0.61 4392weighted avg 0.70 0.70 0.70 4392

Plot for each model vs accuracy score:

comparative plot for model vs accuracy scores

Conclusion

Logistic regression is better model for predicting the results with accuracy of 79.1%

Click here to view my python notebook on Kaggle, and here to view on Linkedin article.