license: apache-2.0
base_model: distilbert-base-uncased
tags:
- generated_from_trainer
model-index:
- name: results
results: []
This model is a fine-tuned version of distilbert-base-uncased, tailored specifically for sentiment analysis. DistilBERT, a distilled version of the more complex BERT model, offers a good balance between performance and resource efficiency, making it ideal for environments where computational resources are limited.
Purpose The primary purpose of this fine-tuned model is to perform sentiment analysis on English movie reviews. It classifies text into positive or negative sentiments based on the content of the review. This model has been trained and evaluated on a subset of the IMDb reviews dataset, making it particularly well-suited for analyzing movie review sentiments.
results
This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.0000
Model description
This model is a fine-tuned version of distilbert-base-uncased, tailored specifically for sentiment analysis. DistilBERT, a distilled version of the more complex BERT model, offers a good balance between performance and resource efficiency, making it ideal for environments where computational resources are limited.
Intended uses & limitations
This model is intended for use in NLP applications where sentiment analysis of English movie reviews is required. It can be easily integrated into applications for analyzing customer feedback, conducting market research, or enhancing user experience by understanding sentiments expressed in text.
The current model is specifically tuned for sentiments in movie reviews and may not perform as well when used on texts from other domains. Additionally, the model's performance might vary depending on the nature of the text, such as informal language or idioms that were not prevalent in the training data.
Training and evaluation data
The model was fine-tuned using the IMDb movie reviews dataset available through HuggingFace's datasets library. This dataset comprises 50,000 highly polar movie reviews split evenly into training and test sets, providing rich text data for training sentiment analysis models. For the purpose of fine-tuning, only 10% of the training set was used to expedite the training process while maintaining a representative sample of the data.
Training procedure
The fine-tuning was performed on Google Colab, utilizing the pre-configured DistilBERT model loaded from HuggingFace's transformers library. The model was fine-tuned for 3 epochs with a batch size of 8 and a learning rate of 5e-5. Special care was taken to maintain the integrity of the tokenization using DistilBERT's default tokenizer, ensuring that the input data was appropriately pre-processed.
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 3
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 0.0003 | 1.0 | 313 | 0.0002 |
| 0.0 | 2.0 | 626 | 0.0000 |
| 0.0 | 3.0 | 939 | 0.0000 |
Framework versions
- Transformers 4.40.2
- Pytorch 2.2.1+cu121
- Datasets 2.19.1
- Tokenizers 0.19.1