bert-large-uncased-finetuned-lowR100-5-uncased-DA-20

This model is a fine-tuned version of bert-large-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.9006

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 30
• eval_batch_size: 30
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 40.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
6.5116	1.0	1	6.5297
6.6949	2.0	2	6.9289
6.0946	3.0	3	7.6464
5.8742	4.0	4	4.8191
5.4365	5.0	5	6.1273
5.171	6.0	6	4.5528
4.4944	7.0	7	4.8541
4.1146	8.0	8	3.4321
3.4689	9.0	9	2.4818
3.6228	10.0	10	2.4444
3.147	11.0	11	1.0668
2.969	12.0	12	3.5394
2.9788	13.0	13	3.1681
2.9108	14.0	14	1.6325
2.9377	15.0	15	2.0480
2.6179	16.0	16	2.6157
2.8978	17.0	17	3.3663
2.6496	18.0	18	2.6341
2.592	19.0	19	2.6462
2.5212	20.0	20	2.2172
2.402	21.0	21	3.3419
2.3146	22.0	22	1.8095
2.5215	23.0	23	2.7622
2.1736	24.0	24	3.9402
2.4366	25.0	25	2.3742
2.1603	26.0	26	2.4520
2.21	27.0	27	3.8185
2.1954	28.0	28	4.0015
2.6556	29.0	29	2.4132
2.3936	30.0	30	3.8690
2.2442	31.0	31	3.7408
2.2486	32.0	32	2.5657
2.5066	33.0	33	3.6632
2.0527	34.0	34	2.9892
2.6207	35.0	35	3.5594
2.296	36.0	36	2.3785
2.4068	37.0	37	3.6126
2.257	38.0	38	1.0477
2.0597	39.0	39	1.5386
2.1702	40.0	40	2.4686

Framework versions

• Transformers 4.26.0
• Pytorch 1.13.1+cu116
• Datasets 2.9.0
• Tokenizers 0.13.2