bert-base-uncased_12112024T130600

This model is a fine-tuned version of google-bert/bert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.2744
  • F1: 0.4792
  • Learning Rate: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 32
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_steps: 600
  • num_epochs: 30
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss F1 Rate
No log 0.9942 86 1.8801 0.0859 0.0000
No log 2.0 173 1.8639 0.1526 0.0000
No log 2.9942 259 1.7658 0.3384 0.0000
No log 4.0 346 1.5763 0.4250 0.0000
No log 4.9942 432 1.3998 0.4554 0.0000
1.5137 6.0 519 1.3179 0.4529 0.0000
1.5137 6.9942 605 1.3344 0.4610 1e-05
1.5137 8.0 692 1.2744 0.4792 0.0000
1.5137 8.9942 778 1.3345 0.4778 0.0000
1.5137 10.0 865 1.3757 0.4794 0.0000
1.5137 10.9942 951 1.4471 0.4857 0.0000
0.6472 12.0 1038 1.5127 0.4885 0.0000
0.6472 12.9942 1124 1.6826 0.4786 0.0000
0.6472 14.0 1211 1.6606 0.4938 0.0000
0.6472 14.9942 1297 1.8626 0.4945 0.0000
0.6472 16.0 1384 1.9157 0.4965 0.0000
0.6472 16.9942 1470 1.9586 0.5034 6e-06
0.179 18.0 1557 1.9809 0.5039 0.0000
0.179 18.9942 1643 2.1634 0.5057 0.0000
0.179 20.0 1730 2.3241 0.4995 0.0000
0.179 20.9942 1816 2.3038 0.5037 0.0000
0.179 22.0 1903 2.3403 0.5091 0.0000
0.179 22.9942 1989 2.3770 0.5044 0.0000
0.0501 24.0 2076 2.5511 0.4936 0.0000
0.0501 24.9942 2162 2.5529 0.4976 0.0000
0.0501 26.0 2249 2.5132 0.5041 7e-07
0.0501 26.9942 2335 2.5330 0.5049 4e-07
0.0501 28.0 2422 2.5368 0.4989 2e-07
0.0248 28.9942 2508 2.5301 0.4973 0.0
0.0248 29.8266 2580 2.5316 0.4973 0.0

Framework versions

  • Transformers 4.44.2
  • Pytorch 2.5.1+cu124
  • Datasets 3.1.0
  • Tokenizers 0.19.1
Downloads last month
12
Safetensors
Model size
109M params
Tensor type
F32
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for Imkaran/bert-base-uncased_12112024T130600

Finetuned
(2307)
this model