pretrained-m-bert-1

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 3.3662
• Validation Loss: 14.3784
• Epoch: 99

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'learning_rate': 1e-04, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Validation Loss	Epoch
10.2573	10.9396	0
7.8912	10.9597	1
6.8354	11.2198	2
6.3659	11.5487	3
6.0787	11.1321	4
5.8528	11.1589	5
5.7689	11.3702	6
5.3858	11.8196	7
5.1194	12.0623	8
5.2013	11.8812	9
5.0820	11.7143	10
4.9735	12.6034	11
5.0704	11.9522	12
5.0098	12.1079	13
5.0869	12.2301	14
4.8156	12.2691	15
4.8344	12.6859	16
4.8416	12.5824	17
4.5567	13.1279	18
4.6677	12.5301	19
4.7248	12.5819	20
4.5435	12.5466	21
4.6501	12.4965	22
4.7348	12.7858	23
4.4649	12.5727	24
4.3665	12.2411	25
4.6434	12.6519	26
4.4239	13.6048	27
4.3893	13.3377	28
4.6514	12.3628	29
4.4743	12.6613	30
4.2777	12.3419	31
4.5667	13.5223	32
4.0886	13.2224	33
4.3032	13.2787	34
4.3670	13.1041	35
4.1487	12.6903	36
4.3331	13.4880	37
4.2907	13.2240	38
4.3252	12.6439	39
4.1121	13.5487	40
4.3273	14.4200	41
4.2030	14.4691	42
4.1795	13.1436	43
4.0424	14.0504	44
4.0158	12.5468	45
4.0108	13.6426	46
3.9515	13.4965	47
3.9743	13.2319	48
4.1075	13.2999	49
4.0501	12.7201	50
3.8606	13.1704	51
3.8056	13.7504	52
3.7682	13.5004	53
4.0676	13.6444	54
4.0957	13.4160	55
4.9373	14.2742	56
4.5111	13.9469	57
4.1604	13.4773	58
3.9956	12.9802	59
4.1232	14.1715	60
3.9857	12.2465	61
4.1082	13.8947	62
3.8659	13.6370	63
3.8396	13.5898	64
3.8220	13.2523	65
3.6864	13.9323	66
3.7541	13.6081	67
3.8218	12.9945	68
3.7251	13.7039	69
3.5017	12.9811	70
3.5342	12.5702	71
3.9520	12.4899	72
3.7465	13.2309	73
3.6003	14.0988	74
3.7954	13.0785	75
3.5654	13.7277	76
3.5591	13.7914	77
3.5355	13.6749	78
3.5903	13.6141	79
3.5371	13.4166	80
3.4502	12.6523	81
3.3372	13.8609	82
3.3071	14.3441	83
3.6932	13.9718	84
3.5619	13.3749	85
3.5016	13.1467	86
3.4279	14.0124	87
3.3140	13.6681	88
3.3575	12.9451	89
3.2268	12.4299	90
3.2001	14.5106	91
3.1390	13.9366	92
3.1230	13.6865	93
3.2337	13.5835	94
3.1397	13.8130	95
3.2095	13.8431	96
2.9553	14.5159	97
3.3319	12.9168	98
3.3662	14.3784	99

Framework versions

• Transformers 4.27.0.dev0
• TensorFlow 2.9.2
• Datasets 2.9.0
• Tokenizers 0.13.2