metadata
license: cc-by-4.0
base_model: deepset/bert-base-cased-squad2
tags:
- generated_from_trainer
model-index:
- name: bert-12
results: []
bert-12
This model is a fine-tuned version of deepset/bert-base-cased-squad2 on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 5.4923
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-06
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 10.7735 | 0.05 | 5 | 11.9148 |
| 10.2311 | 0.09 | 10 | 11.2495 |
| 9.7299 | 0.14 | 15 | 10.3860 |
| 9.6068 | 0.18 | 20 | 9.5703 |
| 8.4179 | 0.23 | 25 | 8.8146 |
| 7.4033 | 0.28 | 30 | 8.1396 |
| 6.9589 | 0.32 | 35 | 7.5510 |
| 6.7006 | 0.37 | 40 | 7.0590 |
| 6.4072 | 0.41 | 45 | 6.6821 |
| 5.9393 | 0.46 | 50 | 6.4175 |
| 5.9838 | 0.5 | 55 | 6.2292 |
| 5.7825 | 0.55 | 60 | 6.1019 |
| 5.4686 | 0.6 | 65 | 6.0218 |
| 5.4427 | 0.64 | 70 | 5.9665 |
| 5.3708 | 0.69 | 75 | 5.9212 |
| 5.5801 | 0.73 | 80 | 5.8795 |
| 5.3894 | 0.78 | 85 | 5.8378 |
| 5.4065 | 0.83 | 90 | 5.7994 |
| 5.2206 | 0.87 | 95 | 5.7603 |
| 5.2044 | 0.92 | 100 | 5.7286 |
| 5.1113 | 0.96 | 105 | 5.7037 |
| 5.0342 | 1.01 | 110 | 5.6786 |
| 4.8333 | 1.06 | 115 | 5.6551 |
| 5.0003 | 1.1 | 120 | 5.6361 |
| 4.7931 | 1.15 | 125 | 5.6200 |
| 4.8148 | 1.19 | 130 | 5.6066 |
| 4.9347 | 1.24 | 135 | 5.5940 |
| 5.0362 | 1.28 | 140 | 5.5797 |
| 4.8616 | 1.33 | 145 | 5.5692 |
| 4.3509 | 1.38 | 150 | 5.5624 |
| 4.6121 | 1.42 | 155 | 5.5595 |
| 4.3364 | 1.47 | 160 | 5.5624 |
| 4.5721 | 1.51 | 165 | 5.5667 |
| 4.4084 | 1.56 | 170 | 5.5684 |
| 4.3097 | 1.61 | 175 | 5.5703 |
| 4.3166 | 1.65 | 180 | 5.5751 |
| 4.257 | 1.7 | 185 | 5.5807 |
| 3.9935 | 1.74 | 190 | 5.5841 |
| 4.1078 | 1.79 | 195 | 5.5872 |
| 4.2609 | 1.83 | 200 | 5.5840 |
| 4.4875 | 1.88 | 205 | 5.5797 |
| 4.1496 | 1.93 | 210 | 5.5747 |
| 4.333 | 1.97 | 215 | 5.5652 |
| 3.9003 | 2.02 | 220 | 5.5594 |
| 3.838 | 2.06 | 225 | 5.5551 |
| 3.8715 | 2.11 | 230 | 5.5525 |
| 4.1982 | 2.16 | 235 | 5.5462 |
| 3.9447 | 2.2 | 240 | 5.5395 |
| 3.8479 | 2.25 | 245 | 5.5352 |
| 3.9253 | 2.29 | 250 | 5.5338 |
| 4.0716 | 2.34 | 255 | 5.5316 |
| 4.0531 | 2.39 | 260 | 5.5295 |
| 4.1593 | 2.43 | 265 | 5.5262 |
| 3.8605 | 2.48 | 270 | 5.5230 |
| 4.1406 | 2.52 | 275 | 5.5170 |
| 4.1568 | 2.57 | 280 | 5.5105 |
| 3.5203 | 2.61 | 285 | 5.5061 |
| 3.8279 | 2.66 | 290 | 5.5042 |
| 4.3043 | 2.71 | 295 | 5.5008 |
| 3.9359 | 2.75 | 300 | 5.4978 |
| 3.6847 | 2.8 | 305 | 5.4954 |
| 3.5765 | 2.84 | 310 | 5.4940 |
| 4.035 | 2.89 | 315 | 5.4927 |
| 3.4865 | 2.94 | 320 | 5.4924 |
| 3.7252 | 2.98 | 325 | 5.4923 |
Framework versions
- Transformers 4.34.0
- Pytorch 2.0.1+cu118
- Datasets 2.14.5
- Tokenizers 0.14.1