distilbert-base-uncased-finetuned-squad

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.3618

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
0.8524	0.2711	1500	1.2645
0.7508	0.5422	3000	1.2776
0.7105	0.8133	4500	1.2793
0.6261	1.0844	6000	1.5100
0.4492	1.3555	7500	1.4754
0.7289	1.6266	9000	1.2560
0.7526	1.8977	10500	1.2356
0.6204	2.1688	12000	1.3780
0.5492	2.4399	13500	1.3522
0.534	2.7110	15000	1.3761
0.5376	2.9821	16500	1.3618

Framework versions

• Transformers 4.41.0.dev0
• Pytorch 2.2.2+cu118
• Datasets 2.19.0
• Tokenizers 0.19.1