bert-base-Maradona

This model is a fine-tuned version of google-bert/bert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.8919

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0002
  • train_batch_size: 32
  • eval_batch_size: 32
  • seed: 42
  • optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 2
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss
1.4175 0.0394 10 1.2472
1.2095 0.0787 20 1.1857
1.133 0.1181 30 1.1394
1.1078 0.1575 40 1.1903
1.1261 0.1969 50 1.1080
1.1278 0.2362 60 1.1327
1.0665 0.2756 70 1.0953
1.0581 0.3150 80 1.1101
1.0518 0.3543 90 1.1255
1.0643 0.3937 100 1.0626
1.0804 0.4331 110 1.0686
1.1146 0.4724 120 1.0215
1.1015 0.5118 130 1.0475
1.0134 0.5512 140 1.0388
0.9956 0.5906 150 1.0563
1.0683 0.6299 160 1.0259
0.9713 0.6693 170 0.9933
1.0103 0.7087 180 1.0096
1.0062 0.7480 190 0.9940
0.9612 0.7874 200 1.0548
1.1625 0.8268 210 1.0181
1.0502 0.8661 220 0.9747
0.9971 0.9055 230 0.9787
0.9128 0.9449 240 0.9965
1.0445 0.9843 250 0.9716
0.9842 1.0236 260 0.9758
0.8422 1.0630 270 1.0168
0.8901 1.1024 280 0.9682
0.9104 1.1417 290 0.9458
0.7868 1.1811 300 0.9196
0.8731 1.2205 310 0.9240
0.7612 1.2598 320 0.9240
0.9062 1.2992 330 0.9240
0.7988 1.3386 340 0.9268
0.7868 1.3780 350 0.9156
0.7878 1.4173 360 0.9161
0.7913 1.4567 370 0.9154
0.8082 1.4961 380 0.9064
0.7385 1.5354 390 0.9012
0.6725 1.5748 400 0.9090
0.7143 1.6142 410 0.9113
0.791 1.6535 420 0.9122
0.7273 1.6929 430 0.9085
0.7976 1.7323 440 0.9028
0.6353 1.7717 450 0.9047
0.8573 1.8110 460 0.8993
0.754 1.8504 470 0.8951
0.7464 1.8898 480 0.8930
0.7193 1.9291 490 0.8924
0.8594 1.9685 500 0.8920

Framework versions

  • Transformers 4.49.0
  • Pytorch 2.3.0+cu121
  • Datasets 3.3.2
  • Tokenizers 0.21.0
Downloads last month
3
Safetensors
Model size
109M params
Tensor type
F32
·
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.

Model tree for YuvrajSingh9886/bert-base-Maradona

Finetuned
(3502)
this model