Sentiment Analysis through LSTMs. Build your own model in less than 50... | by Rohith Gandhi | Towards Data Science

Language bonds humans together. Language acts as a tool for us to convey our thoughts and feelings to another person and with the help of the same tool we are able to understand their thoughts and feelings as well. Most of us start talking from around 18 months old to 2 years. It is not yet completely understood how the human brain is able to grasp such vast amounts of knowledge at such a tender age. But, it has been found that most of the language processing functions happen within the cerebral cortex of the brain.
Humans ourselves are not able to understand how exactly language is processed by our brains. So, is it possible for us to teach a machine to learn our language?? Yes, through extensive research, a lot of methods have been developed that could help machines understand our languages. NLP or Natural Language Processing is the field of study that focuses on the interactions between human language and computers. One subproblem of NLP is sentiment analysis, i.e classifying a statement as positive or negative. What is the use of classifying a statement as positive or negative?? Let’s take an example of Amazon website. On Amazon, it’s users can leave a comment about a product stating whether it was good, bad or it could even be neutral. Now, using a human to read all the comments and obtaining the overall customer feedback on the product would be expensive and time-consuming. Enter our machine learning model. The machine learning model can churn through a vast amount of data, making inferences and classifying the comment. Using this ML model, Amazon can better its products through the customer reviews which would bring in more revenue for the company.
Sentiment analysis isn’t as straightforward as it may seem. If you think that the comments which contain the words “good”, “awesome”, etc can be classified as a positive comment and the comments which the words “bad”, “miserable” etc can be classified as a negative comment, think again. E.x: “Completely lacking in good taste” and “Good for a quick meal but nothing special” represent a negative and neutral feedback respectively even though they have the word “good” in them. Therefore, as I mentioned the task may not be as easy as it may seem. Let’s move on to the data we will be working with.
We are going to look at comments from Amazon products reviews, IMDB movie criticisms and Yelp reviews to build our sentiment analysis model. The data can be downloaded from this link. All the data has been annotated, 0 represents a negative feedback and 1 represents a positive feedback. A few lines of the Amazon data looks similar to the below image.
Let’s write some code!
The data is present in different text files. We open each of this file and read all the lines of text which also includes the labels for each text. We then store them in a list called “lines”.
At each line in the dataset, we have the text and four character spaces after, we have the label for that text(0 or 1). Therefore, we take the first part which has the text and add it to our features(x) and then we take the label, which has ‘\n’ at the end. So, its removed and then added to our list of labels(y).
Keras has a built-in API that makes preparing text for computation easier. The Tokenizer class has 4 attributes which you can use to prepare your features. Take a look at the example below to understand what tokenizer actually does.
## CODEtokenizer = Tokenizer()texts = ["The sun is shining in June!","September is grey.","Life is beautiful in August.","I like it","This and other things?"]tokenizer.fit_on_texts(texts)print(tokenizer.word_index)tokenizer.texts_to_sequences(["June is beautiful and I like it!"])## OUPUT{'sun': 3, 'september': 4, 'june': 5, 'other': 6, 'the': 7, 'and': 8, 'like': 9, 'in': 2, 'beautiful': 11, 'grey': 12, 'life': 17, 'it': 16, 'i': 14, 'is': 1, 'august': 15, 'things': 10, 'shining': 13, 'this': 18}[[5, 1, 11, 8, 14, 9, 16]]
Tokenizer assigns index values to each word in the sentence and a new sentence can represented using this index values. Since the text corpus we are using contains a huge number of different words, we set a upper bound and use only the most recurring 2500 words.
We now convert the text into these number sequences as shown above and pad the sequence of numbers. Since sentences can be of different length, their sequence length will also be different. Therefore, pad_sequences finds the longest sentence and pads all other statements with 0 to match that length.
## Pad Sequences Examplepad_sequences([[1, 2, 3], [3, 4, 5, 6], [7, 8]])array([[0, 1, 2, 3],       [3, 4, 5, 6],       [0, 0, 7, 8]], dtype=int32)
We convert the labels into one hot encoding. This helps the LSTM network in predicting the label for the text. Now, we have prepared our text data, we can split it into training and test samples. 80% of the data is used for training and 20% is kept aside for testing the model.
We now build the model, compile it, train it and test it. The model has an embedding layer. The input sequences are sparse representations of the text because the vocabulary would be huge and a given word would be represented by a large vector. It would be easier for the network to predict if we are able to build some dense representation of the sequence. The word embeddings/dense representation for the 2500 words are derived from training the model by the embedding layer. We then add LSTM and Dense layers to the model. The LSTM cell is responsible for making contextual inferences and aids in predicting whether a sentence is positive or not. The Dense layer outputs the probabilities for each class. I have not given much detail on LSTMs. To know more about them, please refer this blog.
You have just build a sentiment classifier under 50 lines of code.