Edit model card

bert-20

This model is a fine-tuned version of deepset/bert-base-cased-squad2 on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 10.1509

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: constant
  • num_epochs: 4

Training results

Training Loss Epoch Step Validation Loss
11.3191 0.09 5 12.3262
11.4409 0.18 10 12.2742
11.0259 0.27 15 12.2224
11.1517 0.36 20 12.1706
11.1256 0.45 25 12.1189
11.1448 0.55 30 12.0668
10.4156 0.64 35 12.0150
11.0415 0.73 40 11.9641
10.5847 0.82 45 11.9123
10.4663 0.91 50 11.8612
10.2171 1.0 55 11.8098
10.4121 1.09 60 11.7587
10.5127 1.18 65 11.7082
10.7149 1.27 70 11.6574
10.2304 1.36 75 11.6066
10.6459 1.45 80 11.5557
10.3551 1.55 85 11.5044
10.2838 1.64 90 11.4531
10.0924 1.73 95 11.4025
10.295 1.82 100 11.3513
9.8206 1.91 105 11.3005
10.1365 2.0 110 11.2498
10.0496 2.09 115 11.1992
9.8465 2.18 120 11.1489
9.9778 2.27 125 11.0980
10.0708 2.36 130 11.0471
9.6465 2.45 135 10.9962
9.9864 2.55 140 10.9461
9.5175 2.64 145 10.8963
9.9675 2.73 150 10.8461
9.7013 2.82 155 10.7963
9.6324 2.91 160 10.7461
9.7833 3.0 165 10.6960
9.6806 3.09 170 10.6461
9.6208 3.18 175 10.5964
9.3067 3.27 180 10.5468
9.1504 3.36 185 10.4972
9.8082 3.45 190 10.4474
9.3738 3.55 195 10.3978
9.1904 3.64 200 10.3478
9.0302 3.73 205 10.2981
8.8785 3.82 210 10.2490
8.8765 3.91 215 10.1997
9.3 4.0 220 10.1509

Framework versions

  • Transformers 4.34.1
  • Pytorch 2.0.1+cu118
  • Datasets 2.14.5
  • Tokenizers 0.14.1
Downloads last month
3
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for hung200504/bert-20

Finetuned
(28)
this model