badokorach/bert-finetuned-squad-7

This model is a fine-tuned version of badokorach/bert-finetuned-squad-5 on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.0011
• Epoch: 29

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'inner_optimizer': {'class_name': 'AdamWeightDecay', 'config': {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 4e-05, 'decay_steps': 1950, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.01}}, 'dynamic': True, 'initial_scale': 32768.0, 'dynamic_growth_steps': 2000}
• training_precision: mixed_float16

Training results

Train Loss	Epoch
0.0673	0
0.1201	1
0.0502	2
0.0209	3
0.0278	4
0.0358	5
0.0268	6
0.0258	7
0.0212	8
0.0247	9
0.0104	10
0.0101	11
0.0033	12
0.0044	13
0.0185	14
0.0051	15
0.0011	16
0.0043	17
0.0022	18
0.0026	19
0.0019	20
0.0012	21
0.0013	22
0.0009	23
0.0008	24
0.0007	25
0.0016	26
0.0006	27
0.0006	28
0.0011	29

Framework versions

• Transformers 4.31.0
• TensorFlow 2.12.0
• Datasets 2.14.4
• Tokenizers 0.13.3