Edit model card

qa_kor_math

This model is a fine-tuned version of gogamza/kobart-base-v2 on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.3294

Model description

ν•œκ΅­μ–΄ μˆ˜ν•™ 문제λ₯Ό μž…λ ₯ν•˜λ©΄, 문제 μœ ν˜•κ³Ό 문제 μœ ν˜•μ— λŒ€ν•œ μ„€λͺ…, 풀이(μ½”λ“œ), 정닡이 좜λ ₯λ˜λ„λ‘ fine tuning ν–ˆμŠ΅λ‹ˆλ‹€.
문제 μœ ν˜• μ’…λ₯˜λ‘œλŠ” μ‚°μˆ μ—°μ‚°, μˆœμ„œμ •ν•˜κΈ°, μ‘°ν•©ν•˜κΈ°, 수 μ°ΎκΈ°, 크기 비ꡐ, λ„ν˜•μ΄ μžˆμŠ΅λ‹ˆλ‹€.
아직 원인은 잘 λͺ¨λ₯΄κ² μ§€λ§Œ, 정확도가 λ†’μ§€λŠ” μ•Šμ•„λ³΄μž…λ‹ˆλ‹€..

Intended uses & limitations

Training and evaluation data

TUNiB.aiμ—μ„œ github에 κ³΅κ°œν•œ train 데이터 μ…‹μœΌλ‘œ ν•™μŠ΅ν•˜μ˜€μŠ΅λ‹ˆλ‹€.

Training procedure

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 400
  • num_epochs: 20

Training results

Training Loss Epoch Step Validation Loss
No log 0.56 100 3.5725
No log 1.13 200 1.2367
No log 1.69 300 0.7100
No log 2.26 400 0.5420
2.4974 2.82 500 0.5891
2.4974 3.39 600 0.5370
2.4974 3.95 700 0.4738
2.4974 4.52 800 0.4985
2.4974 5.08 900 0.4540
0.3445 5.65 1000 0.4439
0.3445 6.21 1100 0.4261
0.3445 6.78 1200 0.4007
0.3445 7.34 1300 0.3739
0.3445 7.91 1400 0.3937
0.26 8.47 1500 0.3550
0.26 9.04 1600 0.3623
0.26 9.6 1700 0.3944
0.26 10.17 1800 0.3669
0.26 10.73 1900 0.3628
0.217 11.3 2000 0.3703
0.217 11.86 2100 0.3580
0.217 12.43 2200 0.3318
0.217 12.99 2300 0.3199
0.217 13.56 2400 0.3537
0.1916 14.12 2500 0.3198
0.1916 14.69 2600 0.3317
0.1916 15.25 2700 0.3333
0.1916 15.82 2800 0.3280
0.1916 16.38 2900 0.3269
0.1737 16.95 3000 0.3315
0.1737 17.51 3100 0.3346
0.1737 18.08 3200 0.3290
0.1737 18.64 3300 0.3317
0.1737 19.21 3400 0.3282
0.1637 19.77 3500 0.3294

Framework versions

  • Transformers 4.38.2
  • Pytorch 2.2.1+cu121
  • Datasets 2.18.0
  • Tokenizers 0.15.2
Downloads last month
0
Safetensors
Model size
124M params
Tensor type
F32
Β·

Finetuned from