metadata
language: en
tags:
- sentiment-analysis
- transformers
- pytorch
license: apache-2.0
datasets:
- custom-dataset
metrics:
- accuracy
model_name: distilbert-base-uncased-finetuned-sentiment
DistilBERT Base Uncased Fine-tuned for Sentiment Analysis
Model Description
This model is a fine-tuned version of distilbert-base-uncased on a sentiment analysis dataset. It is trained to classify text into positive and negative sentiment categories.
Training Details
The model was fine-tuned on a sentiment analysis dataset using the Hugging Face transformers library. The training parameters are as follows:
- Learning Rate: 2e-5
- Batch Size: 32
- Number of Epochs: 4
- Optimizer: AdamW
- Scheduler: Linear with warmup
- Device: Nvidia T4 GPU
Training and Validation Metrics
| Step | Training Loss | Validation Loss | Accuracy |
|---|---|---|---|
| 400 | 0.389300 | 0.181316 | 93.25% |
| 800 | 0.161900 | 0.166204 | 94.13% |
| 1200 | 0.114600 | 0.200135 | 94.30% |
| 1600 | 0.076300 | 0.211609 | 94.40% |
| 2000 | 0.041600 | 0.225439 | 94.45% |
Final training metrics:
- Global Step: 2000
- Training Loss: 0.156715
- Training Runtime: 1257.5696 seconds
- Training Samples per Second: 50.892
- Training Steps per Second: 1.59
- Total FLOPS: 8477913513984000.0
- Epochs: 4.0
Model Performance
The model achieves an accuracy of approximately 94.45% on the validation set.
Usage
To use this model for sentiment analysis, you can load it using the transformers library:
from transformers import DistilBertTokenizerFast, DistilBertForSequenceClassification
model_name = 'luluw/distilbert-base-uncased-finetuned-sentiment'
tokenizer = DistilBertTokenizerFast.from_pretrained(model_name)
model = DistilBertForSequenceClassification.from_pretrained(model_name)
# Example usage
text = "I love this product!"
inputs = tokenizer(text, return_tensors='pt')
outputs = model(**inputs)
predictions = torch.argmax(outputs.logits, dim=-1)