pretrained-m-bert-200

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 5.6892
• Validation Loss: 15.9999
• Epoch: 199

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'learning_rate': 1e-04, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Validation Loss	Epoch
10.2629	10.9400	0
7.8719	10.8986	1
6.8337	11.4901	2
6.4663	11.6037	3
6.4171	11.5051	4
6.3166	12.1207	5
6.4304	11.7927	6
6.0435	12.1347	7
5.9134	12.1229	8
6.0124	12.0225	9
5.9096	12.4855	10
5.8829	12.7256	11
5.8533	12.3504	12
5.8075	12.7843	13
6.0418	12.6493	14
5.8611	12.4900	15
5.8863	12.7790	16
5.9484	13.0246	17
5.8226	12.9865	18
5.8262	13.1064	19
5.8687	13.1811	20
5.7531	13.2824	21
5.8473	13.2894	22
5.8762	13.1719	23
5.7386	13.0748	24
5.6647	13.3089	25
5.8553	13.5698	26
5.7698	14.1035	27
5.7972	13.6096	28
5.9381	13.1142	29
5.8173	13.1007	30
5.7676	13.6502	31
5.9740	13.5317	32
5.6842	13.7206	33
5.7764	13.5819	34
5.7659	13.4004	35
5.7104	13.6715	36
5.8345	13.5589	37
5.8067	13.6957	38
5.8537	13.6661	39
5.6418	13.8966	40
5.7818	13.7630	41
5.7406	14.1682	42
5.7053	13.8797	43
5.7151	14.1307	44
5.6621	14.1855	45
5.6716	14.1013	46
5.6596	14.2236	47
5.6680	14.0390	48
5.8122	14.0500	49
5.8497	14.0991	50
5.6758	14.5258	51
5.7158	14.2373	52
5.7288	13.9851	53
5.9239	14.2297	54
5.6722	13.6866	55
5.8708	14.2755	56
5.7190	14.4764	57
5.7218	14.1861	58
5.7478	14.3363	59
5.7843	13.9645	60
5.6555	14.1351	61
5.6951	14.5155	62
5.6711	14.4671	63
5.7068	14.4064	64
5.7773	14.5143	65
5.7188	14.6878	66
5.7912	14.3496	67
5.9308	14.4187	68
5.8765	14.6648	69
5.7103	14.3686	70
5.6585	14.3171	71
5.8697	14.2778	72
5.6874	14.1511	73
5.7367	15.0222	74
5.8603	14.2226	75
5.8183	14.6257	76
5.7646	14.5472	77
5.7813	14.4560	78
5.6991	14.1486	79
5.7365	14.5998	80
5.7602	14.3595	81
5.7646	14.4916	82
5.6289	15.1076	83
5.8171	14.7216	84
5.7939	14.9316	85
5.8249	14.6632	86
5.6479	15.2074	87
5.7985	14.9238	88
5.7332	14.4504	89
5.7495	14.2924	90
5.7579	15.3362	91
5.7217	15.0819	92
5.6750	14.9618	93
5.8607	14.6850	94
5.6310	14.9199	95
5.7532	14.8353	96
5.6318	14.9707	97
5.6861	14.8903	98
5.7634	15.3237	99
5.7703	15.0675	100
5.7290	15.5422	101
5.8383	14.9575	102
5.7694	14.2810	103
5.6092	15.5547	104
5.7699	15.2309	105
5.8225	15.0764	106
5.8007	14.8694	107
5.7435	15.2683	108
5.7358	15.3533	109
5.8024	14.8301	110
5.8027	15.3505	111
5.8282	15.1353	112
5.6818	15.3525	113
5.8653	14.7720	114
5.7234	15.2079	115
5.8179	14.9355	116
5.6718	15.2269	117
5.8428	15.1447	118
5.6875	15.2709	119
5.7212	15.1541	120
5.8223	15.2145	121
5.7125	15.2783	122
5.7707	15.6087	123
5.7251	15.1095	124
5.6308	15.2443	125
5.7163	15.7562	126
5.7097	15.5930	127
5.6560	15.1742	128
5.9121	15.0983	129
5.5284	15.4298	130
5.7584	15.5905	131
5.8737	15.3326	132
5.7731	15.6967	133
5.6686	15.2850	134
5.7585	15.2779	135
5.7239	15.6021	136
5.7295	15.3237	137
5.7358	15.3199	138
5.8334	14.8834	139
5.6537	15.6226	140
5.6501	15.2466	141
5.7591	14.9815	142
5.7694	15.3828	143
5.7239	15.4082	144
5.8641	14.8029	145
5.7668	15.4207	146
5.7180	15.8702	147
5.6461	15.7631	148
5.8629	15.2891	149
5.7973	15.9778	150
5.8458	15.4747	151
5.7720	15.9476	152
5.6491	15.2055	153
5.7801	15.3822	154
5.8175	15.7697	155
5.7536	15.2464	156
5.7925	15.4849	157
5.6012	15.5773	158
5.7623	15.7559	159
5.7078	15.7061	160
5.7834	15.5417	161
5.7058	15.3236	162
5.8079	15.1048	163
5.7757	15.2895	164
5.6822	15.9946	165
5.6205	15.8053	166
5.8778	15.9524	167
5.7211	15.5006	168
5.7499	15.7000	169
5.6561	16.1970	170
5.7077	15.7324	171
5.7177	15.8832	172
5.8901	15.2579	173
5.6842	16.1185	174
5.7424	15.8840	175
5.6889	15.5184	176
5.7339	15.9269	177
5.6635	15.8283	178
5.7331	16.0767	179
5.7096	15.7523	180
5.6715	16.0680	181
5.7703	15.6030	182
5.6772	15.6442	183
5.7933	15.6118	184
5.6788	15.5001	185
5.6985	15.4559	186
5.8450	15.5850	187
5.7437	15.9233	188
5.7502	15.8410	189
5.7081	16.0491	190
5.8119	15.3163	191
5.7426	15.7990	192
5.6422	15.9709	193
5.7431	15.3411	194
5.7894	15.5860	195
5.5432	16.2503	196
5.7073	16.0347	197
5.6637	16.2954	198
5.6892	15.9999	199

Framework versions

• Transformers 4.27.0.dev0
• TensorFlow 2.9.2
• Datasets 2.9.0
• Tokenizers 0.13.2