|
--- |
|
language: en |
|
tags: |
|
- sentiment-analysis |
|
- transformers |
|
- pytorch |
|
license: apache-2.0 |
|
datasets: |
|
- custom-dataset |
|
metrics: |
|
- accuracy |
|
model_name: distilbert-base-uncased-finetuned-sentiment |
|
--- |
|
|
|
# DistilBERT Base Uncased Fine-tuned for Sentiment Analysis |
|
|
|
## Model Description |
|
|
|
This model is a fine-tuned version of `distilbert-base-uncased` on a sentiment analysis dataset. It is trained to classify text into positive and negative sentiment categories. |
|
|
|
## Training Details |
|
|
|
The model was fine-tuned on a sentiment analysis dataset using the Hugging Face `transformers` library. The training parameters are as follows: |
|
|
|
- **Learning Rate**: 2e-5 |
|
- **Batch Size**: 32 |
|
- **Number of Epochs**: 4 |
|
- **Optimizer**: AdamW |
|
- **Scheduler**: Linear with warmup |
|
- **Device**: Nvidia T4 GPU |
|
|
|
## Training and Validation Metrics |
|
|
|
| Step | Training Loss | Validation Loss | Accuracy | |
|
|------|---------------|-----------------|----------| |
|
| 400 | 0.389300 | 0.181316 | 93.25% | |
|
| 800 | 0.161900 | 0.166204 | 94.13% | |
|
| 1200 | 0.114600 | 0.200135 | 94.30% | |
|
| 1600 | 0.076300 | 0.211609 | 94.40% | |
|
| 2000 | 0.041600 | 0.225439 | 94.45% | |
|
|
|
Final training metrics: |
|
|
|
- **Global Step**: 2000 |
|
- **Training Loss**: 0.156715 |
|
- **Training Runtime**: 1257.5696 seconds |
|
- **Training Samples per Second**: 50.892 |
|
- **Training Steps per Second**: 1.59 |
|
- **Total FLOPS**: 8477913513984000.0 |
|
- **Epochs**: 4.0 |
|
|
|
## Model Performance |
|
|
|
The model achieves an accuracy of approximately 94.45% on the validation set. |
|
|
|
## Usage |
|
|
|
To use this model for sentiment analysis, you can load it using the `transformers` library: |
|
|
|
```python |
|
from transformers import DistilBertTokenizerFast, DistilBertForSequenceClassification |
|
|
|
model_name = 'luluw/distilbert-base-uncased-finetuned-sentiment' |
|
tokenizer = DistilBertTokenizerFast.from_pretrained(model_name) |
|
model = DistilBertForSequenceClassification.from_pretrained(model_name) |
|
|
|
# Example usage |
|
text = "I love this product!" |
|
inputs = tokenizer(text, return_tensors='pt') |
|
outputs = model(**inputs) |
|
predictions = torch.argmax(outputs.logits, dim=-1) |
|
``` |
|
|
