|
--- |
|
license: apache-2.0 |
|
base_model: distilbert-base-uncased |
|
tags: |
|
- generated_from_trainer |
|
metrics: |
|
- accuracy |
|
- precision |
|
- recall |
|
- f1 |
|
model-index: |
|
- name: results |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# Amazon-Food-Reviews-distilBERT-base for Sentiment Analysis |
|
|
|
## Table of Contents |
|
- [Model Details](#model-details) |
|
- [Uses](#uses) |
|
- [Risks, Limitations and Biases](#risks-limitations-and-biases) |
|
- [Training](#training) |
|
|
|
## Model Details |
|
|
|
**Model Description:** This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on this [Amazon food reviews dataset](https://huggingface.co/datasets/jhan21/amazon-food-reviews-dataset). |
|
|
|
It achieves the following results on the evaluation set: |
|
- Loss: 0.08 |
|
- Accuracy: 0.87 |
|
- Precision: 0.71 |
|
- Recall: 0.77 |
|
- F1: 0.73 |
|
|
|
- **Developed by:** Jiali Han |
|
- **Model Type:** Text Classification |
|
- **Language(s):** English |
|
- **License:** Apache-2.0 |
|
- **Parent Model:** For more details about DistilBERT, please check out [this model card](https://huggingface.co/distilbert-base-uncased). |
|
- **Resources for more information:** |
|
- [Model Documentation](https://huggingface.co/docs/transformers/main/en/model_doc/distilbert#transformers.DistilBertForSequenceClassification) |
|
- [DistilBERT paper](https://arxiv.org/abs/1910.01108) |
|
|
|
## Uses |
|
|
|
#### Direct Use |
|
|
|
This model can be used for sentiment analysis on Amazon food product reviews. |
|
|
|
#### Misuse and Out-of-scope Use |
|
|
|
The model should not be used to create hostile or alienating environments for people intentionally. In addition, the model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model. |
|
|
|
## Risks, Limitations and Biases |
|
|
|
Based on a few experimentations, we observed that this model could produce biased predictions that target underrepresented populations. |
|
|
|
We strongly advise users to thoroughly probe these aspects of their usecases to evaluate this model's risks. We recommend looking at the following bias evaluation datasets as a place to start: [WinoBias](https://huggingface.co/datasets/wino_bias), [WinoGender](https://huggingface.co/datasets/super_glue), [Stereoset](https://huggingface.co/datasets/stereoset). |
|
|
|
|
|
# Training |
|
|
|
#### Training Data |
|
|
|
|
|
The author uses the [Amazon food reviews dataset](https://huggingface.co/datasets/jhan21/amazon-food-reviews-dataset) for the model. |
|
|
|
### Fine-tuning hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 5e-5 |
|
- train_batch_size: 8 |
|
- eval_batch_size: 8 |
|
- seed: 0 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- num_epochs: 5 |
|
|
|
### Training process |
|
|
|
| | Precision | Recall | F1-Score | Support | |
|
|:-----------:|:---------:|:------:|:---------:|:-------:| |
|
| -1 | 0.77 | 0.76 | 0.76 | 851 | |
|
| 0 | 0.38 | 0.62 | 0.47 | 467 | |
|
| 1 | 0.97 | 0.92 | 0.94 | 4985 | |
|
| accuracy | | | 0.87 | 6303 | |
|
| macro avg | 0.71 | 0.77 | 0.73 | 6303 | |
|
|weighted avg | 0.90 | 0.87 | 0.88 | 6303 | |
|
|
|
|
|
### Training process |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Accuracy | Precision | Recall | F1 | |
|
|:-------------:|:-----:|:-----:|:---------------:|:--------:|:---------:|:------:|:------:| |
|
| 0.3730 | 1.00 | 10000 | 0.3706 | 0.8782 | 0.7040 | 0.7657 | 0.7295 | |
|
| 0.3675 | 1.50 | 15000 | 0.3794 | 0.8775 | 0.7107 | 0.7631 | 0.7298 | |
|
| 0.3631 | 2.00 | 20000 | 0.3517 | 0.8805 | 0.7145 | 0.7679 | 0.7226 | |
|
| 0.2732 | 2.50 | 25000 | 0.6240 | 0.8509 | 0.6901 | 0.7784 | 0.7136 | |
|
| 0.2913 | 3.00 | 30000 | 0.4759 | 0.8697 | 0.7132 | 0.7653 | 0.7239 | |
|
| 0.2839 | 3.50 | 35000 | 0.4980 | 0.8755 | 0.7166 | 0.7693 | 0.7311 | |
|
| 0.1983 | 4.00 | 40000 | 0.6700 | 0.8713 | 0.7035 | 0.7767 | 0.7290 | |
|
| 0.2184 | 4.50 | 45000 | 0.5912 | 0.8888 | 0.7147 | 0.7498 | 0.7310 | |
|
| 0.0891 | 4.85 | 48500 | 0.8237 | 0.8731 | 0.7065 | 0.7651 | 0.7258 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.35.2 |
|
- Pytorch 2.1.0+cu118 |
|
- Tokenizers 0.15.0 |
|
|
