pretrained-m-bert-100

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 5.7003
• Validation Loss: 15.3566
• Epoch: 99

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'learning_rate': 1e-04, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Validation Loss	Epoch
10.2669	10.9400	0
7.8880	10.8967	1
6.8580	11.5024	2
6.4321	11.5023	3
6.2235	11.2212	4
6.0038	11.3128	5
5.9881	11.3604	6
5.4409	11.6872	7
5.2113	11.5379	8
5.2660	12.0264	9
5.2330	11.7627	10
5.1121	12.2919	11
5.2126	12.6272	12
5.2086	11.3478	13
5.2459	12.2183	14
5.0035	11.7580	15
4.9613	12.4852	16
5.0312	12.4627	17
5.0073	13.6309	18
5.4284	12.7799	19
5.3100	12.6417	20
5.0765	12.7851	21
5.2276	13.3828	22
5.1986	12.7421	23
4.8935	12.8679	24
4.6959	12.9201	25
5.4161	13.4416	26
5.2459	14.0112	27
5.2781	13.2740	28
5.5104	12.8646	29
5.5024	13.7514	30
5.6284	13.7125	31
5.8452	13.6332	32
5.5767	13.8019	33
5.6444	13.4279	34
5.5551	13.2666	35
5.5421	13.5996	36
5.5246	13.1686	37
5.5233	13.3788	38
5.6011	13.4038	39
5.3695	13.5241	40
5.5061	13.6035	41
5.4534	13.8652	42
5.4222	13.4525	43
5.4408	13.6572	44
5.6683	13.7671	45
5.7137	14.1255	46
5.6777	14.4026	47
5.6776	14.3435	48
5.8337	14.3650	49
5.8583	14.2897	50
5.6849	14.6518	51
5.7112	14.5420	52
5.7281	13.9947	53
5.9154	14.3210	54
5.6742	13.8867	55
5.8674	14.2819	56
5.7128	14.5811	57
5.7091	14.2113	58
5.7479	14.4418	59
5.7632	13.9566	60
5.6443	14.1394	61
5.6794	14.5981	62
5.6450	14.5139	63
5.6935	14.3309	64
5.7443	14.3540	65
5.7014	14.7472	66
5.7407	14.4245	67
5.9023	14.4602	68
5.9222	14.6654	69
5.6813	14.3179	70
5.6505	14.1670	71
5.8407	14.2520	72
5.6683	14.1696	73
5.6880	15.1198	74
5.8254	14.2783	75
5.7758	14.5934	76
5.7180	14.4779	77
5.7348	14.3955	78
5.6680	14.0637	79
5.7029	14.6120	80
5.7088	14.3396	81
5.7215	14.5878	82
5.5987	15.0465	83
5.7613	14.7521	84
5.7670	14.9828	85
5.7954	14.6714	86
5.6080	15.2686	87
5.7493	14.8772	88
5.6884	14.4567	89
5.6932	14.3316	90
5.7152	15.2725	91
5.6548	15.0855	92
5.6196	14.8487	93
5.7889	14.7169	94
5.5958	14.9320	95
5.7047	14.8829	96
5.5637	14.8704	97
5.6375	14.7917	98
5.7003	15.3566	99

Framework versions

• Transformers 4.27.0.dev0
• TensorFlow 2.9.2
• Datasets 2.9.0
• Tokenizers 0.13.2