Sentiment Analysis using Keras & LSTM
By: Rahul Kapoor
Tags: Deep Learning, Sentiment Analysis, Neural Networks, Python, Keras

A Sentiment is a thought, opinion, or idea based on a feeling about a situation, or a way of thinking about something. That something can be about movies, food, restaurant, etc.

With help of user sentiments, we tend to recommend movies, food, etc.

Here we will be focussing on the IMDB movies dataset of 25000 reviews with positive/negative sentiment labels in the training set and an equal amount in the test set. The good thing about this dataset is that it comes with Keras by default and you don't need to download it from another website.

Let’s get started with the implementation.

Import all the base libraries:

import numpy as np

import pandas as pd

import tensorflow as tf

import keras

Import IMDB dataset:

from keras.datasets import imdb (x_train, y_train),(x_test, y_test) = imdb.load_data(num_words=5000)

Confirm the dataset size to be 25000:

print(x_train.shape)

print(x_test.shape)

print(y_train.shape)

print(y_test.shape)

Let’s analyze the training set:

x_train[0]

Output:

[1,

14,

22,

16,

43,

530,

973,

458,

4468,

112,

50,

670,

2,

9,

35,

.

.

.

178,

32]

These are vector representations of word indexes. We need to pad the sequence to a maximum of 500 words. For that Keras provides us with the pad_sequences method:

from keras.preprocessing import sequence x_train = sequence.pad_sequences(x_train, 500)

x_test = sequence.pad_sequences(x_test, 500)

Let us visualize now how it has transformed our training set:

x_train[10]

Output:

array([ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,

0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,

0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,

0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,

0, 0, 0, 0, 0, 0, 1, 785, 189, 438, 47,

110, 142, 7, 6, 2, 120, 4, 236, 378, 7, 153,

19, 87, 108, 141, 17, 1004, 5, 2, 883, 2, 23,

8, 4, 136, 2, 2, 4, 2, 43, 1076, 21, 1407,

419, 5, 2, 120, 91, 682, 189, 2818, 5, 9, 1348,

31, 7, 4, 118, 785, 189, 108, 126, 93, 2, 16,

540, 324, 23, 6, 364, 352, 21, 14, 9, 93, 56,

18, 11, 230, 53, 771, 74, 31, 34, 4, 2834, 7,

4, 22, 5, 14, 11, 471, 9, 2, 34, 4, 321,

487, 5, 116, 15, 2, 4, 22, 9, 6, 2286, 4,

114, 2679, 23, 107, 293, 1008, 1172, 5, 328, 1236, 4,

1375, 109, 9, 6, 132, 773, 2, 1412, 8, 1172, 18,

2, 29, 9, 276, 11, 6, 2768, 19, 289, 409, 4,

2, 2140, 2, 648, 1430, 2, 2, 5, 27, 3000, 1432,

2, 103, 6, 346, 137, 11, 4, 2768, 295, 36, 2,

725, 6, 3208, 273, 11, 4, 1513, 15, 1367, 35, 154,

2, 103, 2, 173, 7, 12, 36, 515, 3547, 94, 2547,

1722, 5, 3547, 36, 203, 30, 502, 8, 361, 12, 8,

989, 143, 4, 1172, 3404, 10, 10, 328, 1236, 9, 6,

55, 221, 2989, 5, 146, 165, 179, 770, 15, 50, 713,

53, 108, 448, 23, 12, 17, 225, 38, 76, 4397, 18,

183, 8, 81, 19, 12, 45, 1257, 8, 135, 15, 2,

166, 4, 118, 7, 45, 2, 17, 466, 45, 2, 4,

22, 115, 165, 764, 2, 5, 1030, 8, 2973, 73, 469,

167, 2127, 2, 1568, 6, 87, 841, 18, 4, 22, 4,

192, 15, 91, 7, 12, 304, 273, 1004, 4, 1375, 1172,

2768, 2, 15, 4, 22, 764, 55, 2, 5, 14, 4233,

2, 4, 1375, 326, 7, 4, 4760, 1786, 8, 361, 1236,

8, 989, 46, 7, 4, 2768, 45, 55, 776, 8, 79,

496, 98, 45, 400, 301, 15, 4, 1859, 9, 4, 155,

15, 66, 2, 84, 5, 14, 22, 1534, 15, 17, 4,

167, 2, 15, 75, 70, 115, 66, 30, 252, 7, 618,

51, 9, 2161, 4, 3130, 5, 14, 1525, 8, 2, 15,

2, 165, 127, 1921, 8, 30, 179, 2532, 4, 22, 9,

906, 18, 6, 176, 7, 1007, 1005, 4, 1375, 114, 4,

105, 26, 32, 55, 221, 11, 68, 205, 96, 5, 4,

192, 15, 4, 274, 410, 220, 304, 23, 94, 205, 109,

9, 55, 73, 224, 259, 3786, 15, 4, 22, 528, 1645,

34, 4, 130, 528, 30, 685, 345, 17, 4, 277, 199,

166, 281, 5, 1030, 8, 30, 179, 4442, 444, 2, 9,

6, 371, 87, 189, 22, 5, 31, 7, 4, 118, 7,

4, 2068, 545, 1178, 829], dtype=int32)

Now we need to build a word_to_id dictionary so that these indexes can be transformed into words for further analysis. In the dictionary, we will provide PAD token to index 0, START token to index 1, and UNK token to index 2. So we have to shift the default indexes by 3 to adjust these tokens.

word_to_id = imdb.get_word_index()

word_to_id = {k:(v+3) for k,v in word_to_id.items()}

word_to_id["<PAD>"] = 0

word_to_id["<START>"] = 1

word_to_id["<UNK>"] = 2

After building word_to_it we need to id_to_word:

id_to_word = {idx:word for word, idx in word_to_id.items()}

Now we can provide id_to_word an index and it will output the word associated with it.

id_to_word[20]

Output:

'movie'

Now we can even read what we have in our training set:

print(" ".join(id_to_word[id] for id in x_train[10]))

Output:

<PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <PAD> <START> french horror cinema has seen something of a <UNK> over the last couple of years with great films such as inside and <UNK> romance <UNK> on to the scene <UNK> <UNK> the <UNK> just slightly but stands head and <UNK> over most modern horror titles and is surely one of the best french horror films ever made <UNK> was obviously shot on a low budget but this is made up for in far more ways than one by the originality of the film and this in turn is <UNK> by the excellent writing and acting that <UNK> the film is a winner the plot focuses on two main ideas prison and black magic the central character is a man named <UNK> sent to prison for <UNK> he is put in a cell with three others the <UNK> insane <UNK> body building <UNK> <UNK> and his retarded boyfriend <UNK> after a short while in the cell together they <UNK> upon a hiding place in the wall that contains an old <UNK> after <UNK> part of it they soon realise its magical powers and realise they may be able to use it to break through the prison walls br br black magic is a very interesting topic and i'm actually quite surprised that there aren't more films based on it as there's so much scope for things to do with it it's fair to say that <UNK> makes the best of it's <UNK> as despite it's <UNK> the film never actually feels <UNK> and manages to flow well throughout director eric <UNK> provides a great atmosphere for the film the fact that most of it takes place inside the central prison cell <UNK> that the film feels very <UNK> and this immensely <UNK> the central idea of the prisoners wanting to use magic to break out of the cell it's very easy to get behind them it's often said that the unknown is the thing that really <UNK> people and this film proves that as the director <UNK> that we can never really be sure of exactly what is round the corner and this helps to <UNK> that <UNK> actually does manage to be quite frightening the film is memorable for a lot of reasons outside the central plot the characters are all very interesting in their own way and the fact that the book itself almost takes on its own character is very well done anyone worried that the film won't deliver by the end won't be disappointed either as the ending both makes sense and manages to be quite horrifying overall <UNK> is a truly great horror film and one of the best of the decade highly recommended viewing

You can try out other indexes as well from the training set to further analyze user reviews. Try to analyze the y_train set as well. You will see 2 labels either 0(negative review) or 1(positive review).