bert-large-uncased-finetuned-lowR100-4-uncased-DA-20

This model is a fine-tuned version of bert-large-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.2624

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 30
• eval_batch_size: 30
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 40.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	1	5.4063
5.9654	2.0	2	6.1913
5.9654	3.0	3	5.8608
5.5864	4.0	4	4.4613
5.5864	5.0	5	5.0257
5.1093	6.0	6	4.7405
5.1093	7.0	7	4.7516
4.3965	8.0	8	3.1646
4.3965	9.0	9	3.7378
3.7825	10.0	10	2.0073
3.7825	11.0	11	5.5725
3.009	12.0	12	3.0077
3.009	13.0	13	2.6288
2.7427	14.0	14	2.8630
2.7427	15.0	15	2.3270
2.4122	16.0	16	3.3092
2.4122	17.0	17	2.7499
2.3707	18.0	18	3.2892
2.3707	19.0	19	2.9385
2.6243	20.0	20	1.5626
2.6243	21.0	21	1.0104
2.1606	22.0	22	3.3464
2.1606	23.0	23	3.9334
2.1419	24.0	24	1.7512
2.1419	25.0	25	2.5855
2.2265	26.0	26	2.2896
2.2265	27.0	27	2.3118
2.2539	28.0	28	2.9997
2.2539	29.0	29	3.0682
2.3866	30.0	30	3.3435
2.3866	31.0	31	3.5848
2.1788	32.0	32	2.4658
2.1788	33.0	33	2.4060
2.0932	34.0	34	2.8497
2.0932	35.0	35	3.8500
1.9799	36.0	36	1.9458
1.9799	37.0	37	2.4557
2.0164	38.0	38	3.3370
2.0164	39.0	39	3.6850
2.1732	40.0	40	1.7986

Framework versions

• Transformers 4.26.0
• Pytorch 1.13.1+cu116
• Datasets 2.9.0
• Tokenizers 0.13.2