DistilBERT Base Uncased Fine-tuned for Sentiment Analysis

Model Description

This model is a fine-tuned version of distilbert-base-uncased on a sentiment analysis dataset. It is trained to classify text into positive and negative sentiment categories.

Training Details

The model was fine-tuned on a sentiment analysis dataset using the Hugging Face transformers library. The training parameters are as follows:

Learning Rate: 2e-5
Batch Size: 32
Number of Epochs: 4
Optimizer: AdamW
Scheduler: Linear with warmup
Device: Nvidia T4 GPU

Training and Validation Metrics

Step	Training Loss	Validation Loss	Accuracy
400	0.389300	0.181316	93.25%
800	0.161900	0.166204	94.13%
1200	0.114600	0.200135	94.30%
1600	0.076300	0.211609	94.40%
2000	0.041600	0.225439	94.45%

Final training metrics:

Global Step: 2000
Training Loss: 0.156715
Training Runtime: 1257.5696 seconds
Training Samples per Second: 50.892
Training Steps per Second: 1.59
Total FLOPS: 8477913513984000.0
Epochs: 4.0

Model Performance

The model achieves an accuracy of approximately 94.45% on the validation set.

Usage

To use this model for sentiment analysis, you can load it using the transformers library:

from transformers import DistilBertTokenizerFast, DistilBertForSequenceClassification

model_name = 'luluw/distilbert-base-uncased-finetuned-sentiment'
tokenizer = DistilBertTokenizerFast.from_pretrained(model_name)
model = DistilBertForSequenceClassification.from_pretrained(model_name)

# Example usage
text = "I love this product!"
inputs = tokenizer(text, return_tensors='pt')
outputs = model(**inputs)
predictions = torch.argmax(outputs.logits, dim=-1)