pad_left

This model is a fine-tuned version of klue/bert-base on the None dataset. It achieves the following results on the evaluation set:

• Exact Match: 56.6667
• F1: 65.2553
• Loss: 3.6763

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Exact Match	F1	Validation Loss
3.2927	0.5	500	40.0	47.4740	1.6933
1.3259	1.0	1000	52.5	62.2490	1.3006
0.8054	1.5	1500	53.75	62.8486	1.1766
0.7301	2.0	2000	53.3333	63.2201	1.1560
0.2873	2.51	2500	52.5	62.0972	1.7569
0.3298	3.01	3000	54.1667	63.9874	1.5193
0.1314	3.51	3500	54.5833	63.9709	2.4847
0.1444	4.01	4000	55.8333	65.8511	2.3190
0.0792	4.51	4500	55.8333	65.2121	2.7640
0.0843	5.01	5000	53.3333	62.7642	3.0308
0.0419	5.51	5500	55.4167	65.3449	3.1388
0.0398	6.01	6000	55.8333	65.1194	3.4126
0.0307	6.51	6500	58.3333	66.8042	3.3642
0.0231	7.01	7000	55.4167	64.5461	3.5422
0.0093	7.52	7500	59.1667	67.8312	3.6604
0.0126	8.02	8000	55.8333	65.3008	3.7195
0.0063	8.52	8500	57.9167	65.9737	3.7285
0.0069	9.02	9000	57.9167	65.9792	3.7144
0.0044	9.52	9500	56.6667	65.2553	3.6763

Framework versions

• Transformers 4.25.1
• Pytorch 1.13.0+cu117
• Datasets 2.7.1
• Tokenizers 0.13.2