|
--- |
|
language: it |
|
thumbnail: https://neuraly.ai/static/assets/images/huggingface/thumbnail.png |
|
tags: |
|
- sentiment |
|
- Italian |
|
license: mit |
|
widget: |
|
- text: Huggingface è un team fantastico! |
|
--- |
|
|
|
# 🤗 + neuraly - Italian BERT Sentiment model |
|
|
|
## Model description |
|
|
|
This model performs sentiment analysis on Italian sentences. It was trained starting from an instance of [bert-base-italian-cased](https://huggingface.co/dbmdz/bert-base-italian-cased), and fine-tuned on an Italian dataset of tweets, reaching 82% of accuracy on the latter one. |
|
|
|
## Intended uses & limitations |
|
|
|
#### How to use |
|
|
|
```python |
|
import torch |
|
from torch import nn |
|
from transformers import AutoTokenizer, AutoModelForSequenceClassification |
|
|
|
# Load the tokenizer |
|
tokenizer = AutoTokenizer.from_pretrained("neuraly/bert-base-italian-cased-sentiment") |
|
# Load the model, use .cuda() to load it on the GPU |
|
model = AutoModelForSequenceClassification.from_pretrained("neuraly/bert-base-italian-cased-sentiment") |
|
|
|
sentence = 'Huggingface è un team fantastico!' |
|
input_ids = tokenizer.encode(sentence, add_special_tokens=True) |
|
|
|
# Create tensor, use .cuda() to transfer the tensor to GPU |
|
tensor = torch.tensor(input_ids).long() |
|
# Fake batch dimension |
|
tensor = tensor.unsqueeze(0) |
|
|
|
# Call the model and get the logits |
|
logits, = model(tensor) |
|
|
|
# Remove the fake batch dimension |
|
logits = logits.squeeze(0) |
|
|
|
# The model was trained with a Log Likelyhood + Softmax combined loss, hence to extract probabilities we need a softmax on top of the logits tensor |
|
proba = nn.functional.softmax(logits, dim=0) |
|
|
|
# Unpack the tensor to obtain negative, neutral and positive probabilities |
|
negative, neutral, positive = proba |
|
``` |
|
|
|
#### Limitations and bias |
|
|
|
A possible drawback (or bias) of this model is related to the fact that it was trained on a tweet dataset, with all the limitations that come with it. The domain is strongly related to football players and teams, but it works surprisingly well even on other topics. |
|
|
|
## Training data |
|
|
|
We trained the model by combining the two tweet datasets taken from [Sentipolc EVALITA 2016](http://www.di.unito.it/~tutreeb/sentipolc-evalita16/data.html). Overall the dataset consists of 45K pre-processed tweets. |
|
|
|
The model weights come from a pre-trained instance of [bert-base-italian-cased](https://huggingface.co/dbmdz/bert-base-italian-cased). A huge "thank you" goes to that team, brilliant work! |
|
|
|
## Training procedure |
|
|
|
#### Preprocessing |
|
|
|
We tried to save as much information as possible, since BERT captures extremely well the semantic of complex text sequences. Overall we removed only **@mentions**, **urls** and **emails** from every tweet and kept pretty much everything else. |
|
|
|
#### Hardware |
|
|
|
- **GPU**: Nvidia GTX1080ti |
|
- **CPU**: AMD Ryzen7 3700x 8c/16t |
|
- **RAM**: 64GB DDR4 |
|
|
|
#### Hyperparameters |
|
|
|
- Optimizer: **AdamW** with learning rate of **2e-5**, epsilon of **1e-8** |
|
- Max epochs: **5** |
|
- Batch size: **32** |
|
- Early Stopping: **enabled** with patience = 1 |
|
|
|
Early stopping was triggered after 3 epochs. |
|
|
|
## Eval results |
|
|
|
The model achieves an overall accuracy on the test set equal to 82% |
|
The test set is a 20% split of the whole dataset. |
|
|
|
## About us |
|
[Neuraly](https://neuraly.ai) is a young and dynamic startup committed to designing AI-driven solutions and services through the most advanced Machine Learning and Data Science technologies. You can find out more about who we are and what we do on our [website](https://neuraly.ai). |
|
|
|
## Acknowledgments |
|
|
|
Thanks to the generous support from the [Hugging Face](https://huggingface.co/) team, |
|
it is possible to download the model from their S3 storage and live test it from their inference API 🤗. |
|
|
