Sentiment Analysis with Pretrained Transformers Using Pytorch | by Raymond Cheng | Towards Data Science

Sentiment Analysis has been a very popular task since the dawn of Natural Language Processing (NLP). It belongs to a subtask or application of text classification, where sentiments or subjective information from different texts are extracted and identified. Today, many businesses around the world use sentiment analysis to understand more deeply their customers and clients by analyzing sentiments across different target groups. It also has wide applications in different sources of information, including product reviews, online social media, survey feedback, etc.
In this article, we will show you how to implement sentiment analysis quickly and effectively using the Transformers library by Huggingface. We will be using pretrained transformers rather than fine-tuning our own, so a low setup cost is needed.
So, let’s jump right into the tutorial!
Step 1: Install Library
Step 2: Import Library
Step 3: Build Sentiment Analysis Pipeline
Step 4: Input Text
Step 5: Perform Semantic Analysis
The library we need to install is the Huggingface Transformers library. To install Transformers, you can simply run:
pip install transformers
Note: Huggingface Transformers requires either Pytorch or Tensorflow to be installed since it relies on either one of them as the backend, thus make sure to have a working version before installing Transformers.
After you have successfully installed Transformers to your local environment, you can create a new Python script and import the Transformers library. Instead of importing the entire library, we introduce the pipeline module within the library that provides a simple API to perform various NLP tasks and hides all code complexity behind its abstraction layer. To import the pipeline module, we can simply do:
from transformers import pipeline
Now after you import the pipeline module, we can start building the sentiment analysis model and tokenizer using the module. To build it, we can do:
sentiment_analysis = pipeline(“sentiment-analysis”)
This will create a pipeline suited for the sentiment analysis task. Wait, but what model and tokenizer are used here, you might ask. Well, by default, the Transformers library uses a DistilBERT model that was fine-tuned on the Stanford Sentiment Treebank v2 (SST2) task from the GLUE Dataset.
If you want to use another model or tokenizer, you can pass them as the model and tokenizer parameters when you instantiate the pipeline.
We’ve built up our pipeline, now it’s time for us to input the text we want to test for its sentiment. Let’s declare two variables for two sentences, one positive and one negative:
pos_text = “I enjoy studying computational algorithms.”neg_text = “I dislike sleeping late everyday.”
Finally, it’s time to classify the sentiments (positive or negative) of our input texts. To perform sentiment analysis, we can run the following:
result = sentiment_analysis(pos_text)[0]print("Label:", result['label'])print("Confidence Score:", result['score'])print()result = sentiment_analysis(neg_text)[0]print("Label:", result['label'])print("Confidence Score:", result['score'])
Output:
Label: POSITIVEConfidence Score: 0.9980718493461609
Label: NEGATIVEConfidence Score: 0.9881595969200134
And Voila! We can now see that the model correctly predicts the two texts into their sentiments, with high confidence scores.
And that’s all for this post! In this short post, we went over how to do sentiment analysis using the simple API provided by the Transformers library. In case you want it, I have attached the Jupyter code of this article down here:
from transformers import pipeline

sentiment_analysis = pipeline("sentiment-analysis")

pos_text = "I enjoy studying computational algorithms."
neg_text = "I dislike sleeping late everyday."

result = sentiment_analysis(pos_text)[0]
print("Label:", result['label'])
print("Confidence Score:", result['score'])
print()

result = sentiment_analysis(neg_text)[0]
print("Label:", result['label'])
print("Confidence Score:", result['score'])

Label: POSITIVE
Confidence Score: 0.9980718493461609

Label: NEGATIVE
Confidence Score: 0.9881595969200134

In case you are wondering how to do text classification other than sentiment analysis for your own custom dataset, you can refer to my previous post on BERT Text Classification. There you will find an in-depth walkthrough and explanation of a text classification example:
towardsdatascience.com
And if you like, feel free to take a look at my other posts, till next time!
towardsdatascience.com
towardsdatascience.com
towardsdatascience.com
towardsdatascience.com
towardsdatascience.com
[1] Transformers Github, Huggingface
[2] Transformers Official Documentation, Huggingface
[3] Pytorch Official Website, Facebook AI Research
[4] Tensorflow Official Website, Google Brain
[5] Sanh, Victor, et al. “DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter.” arXiv preprint arXiv:1910.01108 (2019).
[6] Stanford Sentiment Treebank v2 (SST2), Kaggle
[7] The General Language Understanding Evaluation (GLUE) benchmark, NYU, UW NLP, DeepMind