HasinMDG/XT-Sentiment-XLM-Roberta-Large

General description of the model

Unlike a classical sentiment classifier, this model was built to measure the sentiment towards a particular entity on a particular pre-determined topic

model = ....

text =  "I pity Facebook for their lack of commitment against global warming , I like google for its support of increased education"
# In the previous example we notice that depending on the type of entity (Google or Facebook) and depending on the type of to#pics (education or climate change) we have two types of sentiments

# Predict the sentiment towards Facebook (entity) on Climate change (topic) 
sentiment, probability = model.predict(text, topic="climate change", entity= "Facebook")
# sentiment = "negative

# Predict the sentiment towards Google (entity) on Education (topic) 
sentiment, probability = model.predict(text, topic="climate change", entity= "Facebook")
# Sentiment = "positive"

# Predict the sentiment towards Google (entity) on Climate Change (topic) 
sentiment, probability = model.predict(text, topic="climate change", entity= "Facebook")
# Sentiment = "neutral" / "not_found"

# Predict the sentiment towards Facebook (entity) on Education (topic) 
sentiment, probability = model.predict(text, topic="climate change", entity= "Facebook")
# Sentiment = "neutral" / "not_found"

Training

This is a SetFit model that can be used for sentiment classification. The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.
The Training data can be downloaded from here

Usage and Inference

For a global overview of the pipeline used for inference please refer to this colab notebook

Model Performance

The performances of the model on our internal test set are:

Accuracy: 0.68
Balanced_Accuracy: 0.45
MCC: 0.37
F1: 0.49

Potential weakness of the model

As the model has been trained on data of short length, it is difficult to predict how the model will behave on long texts
Although the model is robust to typos and able to deal with synonyms, the entities and topics must be as explicit as possible.
The model may have difficulties to detect very abstract and complex topics, a fine tuning of the model can solve this problem
The model may have difficulty in capturing elements that are very specific to a given context

BibTeX entry and citation info

author = {HasiMichael, Solofo, Bruce, Sitwala},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Sentiment Classification toward Entity and Topics},
year = {2023/04},
version = {0}