QA_BERT_200_epoch

This model was trained from scratch on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.2346

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 160
• eval_batch_size: 80
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 17

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	2	5.4257
No log	2.0	4	4.7619
No log	3.0	6	4.1887
No log	4.0	8	3.8113
No log	5.0	10	3.6620
No log	6.0	12	3.6115
No log	7.0	14	3.5094
No log	8.0	16	3.4270
No log	9.0	18	3.3470
No log	10.0	20	3.2998
No log	11.0	22	3.2984
No log	12.0	24	3.3130
No log	13.0	26	3.3005
No log	14.0	28	3.2661
No log	15.0	30	3.2345
No log	16.0	32	3.2227
No log	17.0	34	3.2346

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu121
• Datasets 2.14.7
• Tokenizers 0.15.0