pretrained-bert-uncased-50

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 5.6891
• Validation Loss: 13.0813
• Epoch: 49

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'learning_rate': 1e-04, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Validation Loss	Epoch
8.9180	9.5765	0
7.0631	9.6436	1
6.5662	10.4855	2
6.1181	10.4295	3
6.2069	10.4589	4
5.9769	11.0551	5
6.1633	10.8653	6
5.9430	11.3191	7
5.7405	10.9468	8
5.8629	11.1128	9
5.8546	11.6032	10
5.7616	12.0396	11
5.7221	11.4718	12
5.8037	11.6265	13
5.8047	11.7407	14
5.6255	11.9321	15
5.7123	11.9664	16
5.7439	12.3851	17
5.6358	11.7695	18
5.6334	12.1840	19
5.7475	12.3386	20
5.6651	12.1824	21
5.6864	12.1818	22
5.7437	12.1632	23
5.5457	12.3682	24
5.4805	12.2731	25
5.7177	12.7105	26
5.6454	12.3077	27
5.6164	12.4352	28
5.8538	12.2957	29
5.6449	12.4987	30
5.5644	12.6280	31
5.8275	12.5619	32
5.5706	13.0127	33
5.6039	12.4849	34
5.6839	12.2682	35
5.6085	12.5530	36
5.5826	12.4190	37
5.5802	12.8276	38
5.6272	12.9266	39
5.4752	13.0355	40
5.5738	12.8739	41
5.6231	13.1363	42
5.5497	12.6590	43
5.6278	13.0785	44
5.4599	13.0727	45
5.5782	13.2001	46
5.5343	13.4125	47
5.4846	13.3727	48
5.6891	13.0813	49

Framework versions

• Transformers 4.27.0.dev0
• TensorFlow 2.9.2
• Datasets 2.9.0
• Tokenizers 0.13.2