bert-model-disaster-tweets-classification

This model is a fine-tuned version of bert-base-uncased on the Natural-Language-Processing-with-Disaster-Tweets dataset. It achieves the following results on the evaluation set:

Accuracy: 0.82
F1 Score: 0.82

Model description

Load BertForSequenceClassification, the pretrained BERT model with a single linear classification layer on top, using an optimizer : incorporates weight decay, which is a regularization technique that helps prevent overfitting during training.

Intended uses & limitations

Use to classify if a tweet represents a disaster or not.

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 32
eval_batch_size: 32
seed: 42
optimizer: Adam with epsilon = 1e-8.
lr_scheduler_type: linear
num_epochs: 3

Training results

Epoch	Average training loss	Training epoch	Accuracy	F1
1.0	0.47	0:00:49	0.82	0.82
2.0	0.36	0:00:36	0.82	0.82
3.0	0.29	0:00:51	0.82	0.82

Framework versions

Transformers 4.28.1
Pytorch 2.0.0+cu118
Datasets 2.12.0
Tokenizers 0.13.3