pretrained-m-bert-300

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 5.8273
• Validation Loss: 15.6623
• Epoch: 299

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'learning_rate': 1e-04, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Validation Loss	Epoch
10.2479	10.9372	0
7.7731	10.9191	1
6.8702	11.5201	2
6.4849	11.6086	3
6.3725	11.5271	4
6.3243	12.1350	5
6.4515	11.7665	6
6.0675	12.1761	7
5.9322	12.1155	8
6.0672	12.0390	9
5.9976	12.5114	10
5.9208	12.7953	11
5.9503	12.4924	12
5.9696	12.7799	13
6.0537	12.3489	14
5.8556	12.5165	15
5.8976	12.8338	16
5.9458	13.0800	17
5.8258	12.9819	18
5.8284	13.0523	19
5.8739	13.0829	20
5.7537	13.1990	21
5.8624	13.2222	22
5.8871	13.1393	23
5.7382	13.0271	24
5.6791	13.3209	25
5.8651	13.5971	26
5.7795	14.0682	27
5.7961	13.5632	28
5.9525	13.0326	29
5.8251	13.0935	30
5.7616	13.5397	31
5.9793	13.4677	32
5.6852	13.6610	33
5.7826	13.6501	34
5.7675	13.3981	35
5.7075	13.6568	36
5.8363	13.5032	37
5.8045	13.6162	38
5.8582	13.5919	39
5.6427	13.8740	40
5.7807	13.7311	41
5.7421	14.1702	42
5.7074	13.8185	43
5.7145	14.0385	44
5.6605	14.0947	45
5.6647	13.9634	46
5.6628	14.1416	47
5.6652	13.9625	48
5.8173	14.0109	49
5.8535	14.0783	50
5.6777	14.4908	51
5.7189	14.2846	52
5.7306	13.9430	53
5.9265	14.2692	54
5.6752	13.7434	55
5.8745	14.2234	56
5.7229	14.4659	57
5.7215	14.0766	58
5.7540	14.3406	59
5.7831	13.9421	60
5.6559	14.0940	61
5.6964	14.4394	62
5.6707	14.4002	63
5.7088	14.3143	64
5.7738	14.3808	65
5.7194	14.6182	66
5.7911	14.2589	67
5.9282	14.3536	68
5.8769	14.5976	69
5.7150	14.3358	70
5.6573	14.2675	71
5.8684	14.2212	72
5.6871	14.0757	73
5.7349	14.9877	74
5.8587	14.1604	75
5.8195	14.4759	76
5.7681	14.4587	77
5.7803	14.4228	78
5.6986	14.1285	79
5.7369	14.5417	80
5.7565	14.2100	81
5.7648	14.4228	82
5.6307	15.0572	83
5.8166	14.6594	84
5.7945	14.9603	85
5.8273	14.6196	86
5.6483	15.2973	87
5.7982	14.9318	88
5.7286	14.4151	89
5.7488	14.2480	90
5.7564	15.2868	91
5.7200	14.9984	92
5.6758	14.8934	93
5.8600	14.6392	94
5.6302	14.9115	95
5.7530	14.8292	96
5.6311	14.9683	97
5.6845	14.8707	98
5.7639	15.2866	99
5.7692	15.1005	100
5.7279	15.5260	101
5.8349	14.8966	102
5.7720	14.2529	103
5.6082	15.5972	104
5.7725	15.1931	105
5.8239	15.1119	106
5.7973	14.8203	107
5.7439	15.2762	108
5.7344	15.2897	109
5.8002	14.8071	110
5.7978	15.3206	111
5.8302	15.1250	112
5.6829	15.3822	113
5.8658	14.7853	114
5.7236	15.1413	115
5.8151	14.9191	116
5.6697	15.2308	117
5.8450	15.2055	118
5.6843	15.3117	119
5.7215	15.1254	120
5.8230	15.1992	121
5.7106	15.2795	122
5.7720	15.6248	123
5.7214	15.0411	124
5.6302	15.2897	125
5.7151	15.7383	126
5.7107	15.5989	127
5.6569	15.2202	128
5.9129	15.1588	129
5.5289	15.4879	130
5.7570	15.5103	131
5.8748	15.3842	132
5.7679	15.6996	133
5.6655	15.2690	134
5.7573	15.2401	135
5.7238	15.5996	136
5.7273	15.3198	137
5.7344	15.3389	138
5.8311	14.8744	139
5.6549	15.6956	140
5.6496	15.2694	141
5.7590	15.0076	142
5.7703	15.3850	143
5.7206	15.4296	144
5.8623	14.8546	145
5.7601	15.4164	146
5.7175	15.8795	147
5.6459	15.8282	148
5.8591	15.3127	149
5.7940	16.0000	150
5.8439	15.5051	151
5.7669	15.9199	152
5.6481	15.2306	153
5.7793	15.4377	154
5.8167	15.7849	155
5.7556	15.2991	156
5.7905	15.5514	157
5.5980	15.6595	158
5.7624	15.7794	159
5.7073	15.7131	160
5.7823	15.6013	161
5.6993	15.3206	162
5.8054	15.1585	163
5.7734	15.3361	164
5.6832	16.0706	165
5.6192	15.7624	166
5.8735	15.9157	167
5.7212	15.5399	168
5.7479	15.7155	169
5.6542	16.2107	170
5.7076	15.7150	171
5.7149	15.8730	172
5.8877	15.2373	173
5.6803	16.1623	174
5.7420	15.9171	175
5.6912	15.5799	176
5.7350	16.0120	177
5.6631	15.9157	178
5.7305	16.1250	179
5.7077	15.8018	180
5.6688	16.1011	181
5.7675	15.6628	182
5.6747	15.6886	183
5.7921	15.6053	184
5.6793	15.5329	185
5.6993	15.4673	186
5.8451	15.6634	187
5.7389	15.9733	188
5.7486	15.8548	189
5.7089	16.1267	190
5.8106	15.4471	191
5.7402	15.8568	192
5.6393	15.9586	193
5.7403	15.2678	194
5.7854	15.5638	195
5.5414	16.1871	196
5.7082	15.9706	197
5.6636	16.2550	198
5.6875	15.9385	199
5.7139	15.6730	200
5.6601	15.4174	201
5.6422	16.1655	202
5.7642	16.3103	203
5.7039	16.4020	204
5.7237	15.8775	205
5.7529	15.7237	206
5.6827	16.1514	207
5.7591	16.0905	208
5.7899	15.6417	209
5.7775	16.3878	210
5.6634	15.9944	211
5.5958	16.1042	212
5.8629	16.6206	213
5.7548	16.3826	214
5.7512	16.2234	215
5.6905	16.5029	216
5.6434	16.8345	217
5.6728	15.8749	218
5.7253	16.1679	219
5.6529	15.9138	220
5.6542	16.4299	221
5.6646	15.9442	222
5.7054	16.3624	223
5.7083	16.1256	224
5.8134	15.8207	225
5.7805	16.2750	226
5.7037	15.9758	227
5.7653	16.2336	228
5.7890	16.4635	229
5.7060	16.2425	230
5.7508	16.2569	231
5.6349	16.4228	232
5.7062	16.5237	233
5.7277	16.4191	234
5.7827	16.0735	235
5.7090	16.3830	236
5.6960	16.3506	237
5.7367	15.9862	238
5.7863	16.2742	239
5.5916	16.3640	240
5.6753	16.7890	241
5.6915	16.5041	242
5.7292	16.4998	243
5.7814	16.1040	244
5.6399	16.4167	245
5.6281	16.1772	246
5.7067	16.5245	247
5.7268	16.3465	248
5.7664	16.5136	249
5.7020	16.1559	250
5.6693	16.8744	251
5.6625	15.9549	252
5.6282	16.4120	253
5.6190	15.9476	254
5.6562	16.2114	255
5.6690	16.2859	256
5.7533	16.3209	257
5.7191	16.3224	258
5.8181	16.1149	259
5.6598	16.2559	260
5.6762	16.5949	261
5.6452	16.2653	262
5.6691	16.2993	263
5.7951	16.0316	264
5.8137	16.3896	265
5.7124	16.3996	266
5.7853	16.6237	267
5.7931	15.6052	268
5.7788	16.5983	269
5.7472	16.0878	270
5.6607	16.6207	271
5.8085	16.5659	272
5.7699	16.1165	273
5.6865	16.3090	274
5.7237	16.1727	275
5.8241	16.1545	276
5.6519	16.5434	277
5.6718	16.4884	278
5.6988	16.4953	279
5.7020	16.8616	280
5.7338	16.3847	281
5.6695	16.4040	282
5.6916	16.3199	283
5.7519	15.6585	284
5.7317	16.4947	285
5.8143	15.9633	286
5.6979	16.5859	287
5.7405	16.5161	288
5.7338	16.4144	289
5.5844	16.5315	290
5.6871	16.4282	291
5.8713	15.5593	292
5.6710	15.8436	293
5.7074	16.4072	294
5.6212	16.4969	295
5.7022	16.3911	296
5.6552	16.8670	297
5.7888	16.2774	298
5.8273	15.6623	299

Framework versions

• Transformers 4.27.0.dev0
• TensorFlow 2.9.2
• Datasets 2.9.0
• Tokenizers 0.13.2