bert-large-cased-finetuned-lowR100-2-cased-DA-20

This model is a fine-tuned version of bert-large-cased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.7801

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 30
• eval_batch_size: 30
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 40.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
6.4515	1.0	1	8.1791
6.4671	2.0	2	6.0155
6.533	3.0	3	5.9784
5.8654	4.0	4	5.2092
5.5458	5.0	5	6.1062
5.1806	6.0	6	5.0913
4.8797	7.0	7	4.3025
4.6975	8.0	8	4.8598
4.2859	9.0	9	4.2301
4.3584	10.0	10	4.0683
4.0203	11.0	11	2.7986
3.977	12.0	12	4.1575
3.4077	13.0	13	3.6507
3.313	14.0	14	2.8674
3.0962	15.0	15	2.5103
2.8883	16.0	16	3.1318
2.9623	17.0	17	2.1316
2.5544	18.0	18	2.7741
2.9957	19.0	19	2.9045
2.749	20.0	20	2.8824
2.291	21.0	21	2.7450
2.3373	22.0	22	2.3774
2.6506	23.0	23	2.5515
2.6736	24.0	24	2.2106
2.3845	25.0	25	2.3166
2.3762	26.0	26	2.3221
2.4184	27.0	27	2.8996
2.6826	28.0	28	2.1793
2.4678	29.0	29	2.4268
2.2998	30.0	30	1.8153
2.7085	31.0	31	2.4401
2.1231	32.0	32	3.3329
2.1349	33.0	33	1.9675
2.4647	34.0	34	3.0172
2.3552	35.0	35	1.8550
2.2843	36.0	36	2.7737
2.2164	37.0	37	3.4890
2.2118	38.0	38	3.4251
2.3133	39.0	39	2.6806
1.9773	40.0	40	2.7801

Framework versions

• Transformers 4.26.0
• Pytorch 1.13.1+cu116
• Datasets 2.9.0
• Tokenizers 0.13.2