mobilebert-uncased-squad-v2-16-11-2

This model is a fine-tuned version of csarron/mobilebert-uncased-squad-v2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 16.1284

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	489	2.6193
1.4634	2.0	978	3.0847
1.2884	3.0	1467	3.2589
1.1511	4.0	1956	3.9182
1.0809	5.0	2445	3.7122
1.008	6.0	2934	4.5737
0.9048	7.0	3423	5.2430
0.7411	8.0	3912	5.4474
0.6668	9.0	4401	5.9275
0.557	10.0	4890	7.8979
0.4912	11.0	5379	7.8582
0.409	12.0	5868	8.1236
0.3293	13.0	6357	9.7170
0.3408	14.0	6846	10.1125
0.2514	15.0	7335	10.8043
0.2042	16.0	7824	11.1361
0.201	17.0	8313	12.5571
0.1846	18.0	8802	13.4892
0.1582	19.0	9291	13.4029
0.1185	20.0	9780	14.8577
0.1048	21.0	10269	15.3951
0.1258	22.0	10758	15.3019
0.0763	23.0	11247	15.5361
0.0684	24.0	11736	15.8837
0.0667	25.0	12225	16.1284

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu118
• Datasets 2.15.0
• Tokenizers 0.15.0