bert-large-cased-finetuned-lowR100-3-cased-DA-20

This model is a fine-tuned version of bert-large-cased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.5250

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 30
• eval_batch_size: 30
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 40.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	1	6.8611
6.5268	2.0	2	8.5069
6.5268	3.0	3	6.4383
6.3552	4.0	4	5.2540
6.3552	5.0	5	6.2490
5.5713	6.0	6	5.8587
5.5713	7.0	7	5.6369
5.0248	8.0	8	5.1667
5.0248	9.0	9	4.8407
4.364	10.0	10	5.0590
4.364	11.0	11	4.8647
3.6607	12.0	12	3.3072
3.6607	13.0	13	3.4963
3.3901	14.0	14	4.0039
3.3901	15.0	15	3.5993
3.1245	16.0	16	2.2179
3.1245	17.0	17	1.6414
3.1906	18.0	18	3.1965
3.1906	19.0	19	3.1463
2.7243	20.0	20	3.1866
2.7243	21.0	21	1.0648
2.944	22.0	22	3.2413
2.944	23.0	23	3.1838
2.7114	24.0	24	3.8036
2.7114	25.0	25	2.2897
2.4176	26.0	26	3.6953
2.4176	27.0	27	3.3176
2.4277	28.0	28	2.9940
2.4277	29.0	29	3.0186
2.4099	30.0	30	3.0385
2.4099	31.0	31	1.9323
2.2141	32.0	32	2.2952
2.2141	33.0	33	3.5302
2.4007	34.0	34	3.7787
2.4007	35.0	35	3.3718
2.2619	36.0	36	2.2895
2.2619	37.0	37	2.7433
2.4834	38.0	38	3.5129
2.4834	39.0	39	1.7792
2.122	40.0	40	2.5250

Framework versions

• Transformers 4.26.0
• Pytorch 1.13.1+cu116
• Datasets 2.9.0
• Tokenizers 0.13.2