bert-large-cased-finetuned-cased-DA-20

This model is a fine-tuned version of bert-large-cased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.9537

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
1.8306	1.0	225	1.5331
1.5024	2.0	450	1.3427
1.3769	3.0	675	1.2441
1.2924	4.0	900	1.2326
1.2491	5.0	1125	1.1679
1.1969	6.0	1350	1.1379
1.1505	7.0	1575	1.1036
1.1198	8.0	1800	1.0733
1.0925	9.0	2025	1.0709
1.0707	10.0	2250	1.0450
1.0465	11.0	2475	1.0228
1.0329	12.0	2700	1.0219
1.0104	13.0	2925	0.9946
0.9924	14.0	3150	1.0020
0.9882	15.0	3375	0.9802
0.9836	16.0	3600	0.9773
0.9725	17.0	3825	0.9729
0.9633	18.0	4050	0.9558
0.9601	19.0	4275	0.9707
0.9553	20.0	4500	0.9555

Framework versions

• Transformers 4.26.0
• Pytorch 1.13.1+cu116
• Datasets 2.9.0
• Tokenizers 0.13.2