Astonish24/bert-finetuned-squad

This model is a fine-tuned version of Astonish24/bert-finetuned-squad on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.0002
• Epoch: 199

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'inner_optimizer': {'module': 'transformers.optimization_tf', 'class_name': 'AdamWeightDecay', 'config': {'name': 'AdamWeightDecay', 'learning_rate': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 800, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'decay': 0.0, 'beta_1': 0.8999999761581421, 'beta_2': 0.9990000128746033, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.01}, 'registered_name': 'AdamWeightDecay'}, 'dynamic': True, 'initial_scale': 32768.0, 'dynamic_growth_steps': 2000}
• training_precision: mixed_float16

Training results

Train Loss	Epoch
0.0038	0
0.0079	1
0.0025	2
0.0324	3
0.0078	4
0.0281	5
0.0088	6
0.0088	7
0.0220	8
0.0128	9
0.0236	10
0.0102	11
0.0103	12
0.0017	13
0.0038	14
0.0052	15
0.0056	16
0.0269	17
0.0067	18
0.0064	19
0.0036	20
0.0015	21
0.0016	22
0.0024	23
0.0123	24
0.0286	25
0.0179	26
0.0069	27
0.0139	28
0.0056	29
0.0028	30
0.0055	31
0.0016	32
0.0020	33
0.0034	34
0.0016	35
0.0012	36
0.0012	37
0.0007	38
0.0011	39
0.0025	40
0.0007	41
0.0010	42
0.0008	43
0.0007	44
0.0012	45
0.0007	46
0.0006	47
0.0004	48
0.0005	49
0.0007	50
0.0004	51
0.0003	52
0.0007	53
0.0003	54
0.0004	55
0.0005	56
0.0004	57
0.0003	58
0.0003	59
0.0004	60
0.0003	61
0.0002	62
0.0003	63
0.0019	64
0.0018	65
0.0021	66
0.0008	67
0.0005	68
0.0011	69
0.0004	70
0.0004	71
0.0005	72
0.0007	73
0.0003	74
0.0005	75
0.0005	76
0.0003	77
0.0003	78
0.0004	79
0.0005	80
0.0003	81
0.0003	82
0.0002	83
0.0003	84
0.0002	85
0.0003	86
0.0002	87
0.0004	88
0.0002	89
0.0003	90
0.0003	91
0.0024	92
0.0014	93
0.0021	94
0.0018	95
0.0008	96
0.0009	97
0.0006	98
0.0005	99
0.0003	100
0.0005	101
0.0002	102
0.0005	103
0.0004	104
0.0006	105
0.0004	106
0.0003	107
0.0004	108
0.0002	109
0.0003	110
0.0002	111
0.0002	112
0.0005	113
0.0002	114
0.0004	115
0.0002	116
0.0002	117
0.0002	118
0.0002	119
0.0002	120
0.0001	121
0.0002	122
0.0002	123
0.0001	124
0.0003	125
0.0002	126
0.0002	127
0.0002	128
0.0002	129
0.0002	130
0.0002	131
0.0002	132
0.0002	133
0.0002	134
0.0001	135
0.0002	136
0.0002	137
0.0002	138
0.0003	139
0.0001	140
0.0002	141
0.0003	142
0.0001	143
0.0003	144
0.0003	145
0.0002	146
0.0002	147
0.0002	148
0.0002	149
0.0002	150
0.0001	151
0.0002	152
0.0001	153
0.0002	154
0.0002	155
0.0001	156
0.0001	157
0.0001	158
0.0002	159
0.0003	160
0.0002	161
0.0001	162
0.0002	163
0.0002	164
0.0001	165
0.0001	166
0.0001	167
0.0001	168
0.0002	169
0.0002	170
0.0001	171
0.0003	172
0.0001	173
0.0001	174
0.0001	175
0.0007	176
0.0004	177
0.0001	178
0.0002	179
0.0001	180
0.0001	181
0.0002	182
0.0007	183
0.0002	184
0.0002	185
0.0002	186
0.0001	187
0.0001	188
0.0001	189
0.0002	190
0.0002	191
0.0001	192
0.0002	193
0.0002	194
0.0001	195
0.0001	196
0.0002	197
0.0002	198
0.0002	199

Framework versions

• Transformers 4.47.1
• TensorFlow 2.17.1
• Datasets 3.2.0
• Tokenizers 0.21.0