README.md

metadata

license: mit
base_model: gogamza/kobart-base-v2
tags:
  - generated_from_trainer
model-index:
  - name: qa_kor_math
    results: []

qa_kor_math

This model is a fine-tuned version of gogamza/kobart-base-v2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.3294

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 400
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.56	100	3.5725
No log	1.13	200	1.2367
No log	1.69	300	0.7100
No log	2.26	400	0.5420
2.4974	2.82	500	0.5891
2.4974	3.39	600	0.5370
2.4974	3.95	700	0.4738
2.4974	4.52	800	0.4985
2.4974	5.08	900	0.4540
0.3445	5.65	1000	0.4439
0.3445	6.21	1100	0.4261
0.3445	6.78	1200	0.4007
0.3445	7.34	1300	0.3739
0.3445	7.91	1400	0.3937
0.26	8.47	1500	0.3550
0.26	9.04	1600	0.3623
0.26	9.6	1700	0.3944
0.26	10.17	1800	0.3669
0.26	10.73	1900	0.3628
0.217	11.3	2000	0.3703
0.217	11.86	2100	0.3580
0.217	12.43	2200	0.3318
0.217	12.99	2300	0.3199
0.217	13.56	2400	0.3537
0.1916	14.12	2500	0.3198
0.1916	14.69	2600	0.3317
0.1916	15.25	2700	0.3333
0.1916	15.82	2800	0.3280
0.1916	16.38	2900	0.3269
0.1737	16.95	3000	0.3315
0.1737	17.51	3100	0.3346
0.1737	18.08	3200	0.3290
0.1737	18.64	3300	0.3317
0.1737	19.21	3400	0.3282
0.1637	19.77	3500	0.3294

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2