hw_midm_7B_nsmc / README.md
cxoijve's picture
Update README.md
544be6c
metadata
library_name: peft
base_model: KT-AI/midm-bitext-S-7B-inst-v1
datasets:
  - nsmc

Model Description

  • NSMC ๋ฐ์ดํ„ฐ์— ๋Œ€ํ•ด KT-AI/midm-bitext-S-7B-inst-v1 ๋ฏธ์„ธํŠœ๋‹
  • ์˜ํ™” ๋ฆฌ๋ทฐ ํ…์ŠคํŠธ๋ฅผ ํ”„๋กฌํ”„ํŠธ์— ํฌํ•จํ•˜์—ฌ ๋ชจ๋ธ์— ์ž…๋ ฅํ•˜๋ฉด '๊ธ์ •' ๋˜๋Š” '๋ถ€์ •'์ด๋ผ๊ณ  ์˜ˆ์ธก ํ…์ŠคํŠธ๋ฅผ ์ง์ ‘ ์ƒ์„ฑ
  • NSMC์˜ train ์Šคํ”Œ๋ฆฟ ์ƒ์œ„ 2,000๊ฐœ ์ด์ƒ์˜ ์ƒ˜ํ”Œ์„ ํ•™์Šต์— ์‚ฌ์šฉ
  • test ์Šคํ”Œ๋ฆฟ ์ƒ์œ„ 1,000๊ฐœ์˜ ์ƒ˜ํ”Œ๋งŒ ์ธก์ •

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 1
  • eval_batch_size: 1
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 2
  • optimizer: adam_beta1=0.9, adam_beta2=0.999, adam_epsilon=1e-08,
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.03
  • training_args.logging_steps: 50
  • training_args.max_steps : 300
  • trainable params: 16,744,448 || all params: 7,034,347,520 || trainable%: 0.23803839591934178

Training Results

TrainOutput( global_step=300, training_loss=2.666887741088867, metrics={'train_runtime': 961.226, 'train_samples_per_second': 0.624, 'train_steps_per_second': 0.312, 'total_flos': 9315508499251200.0, 'train_loss': 2.666887741088867, 'epoch': 0.3})

Accuracy

Midm: ์ •ํ™•๋„ 0.88

TP TN
PP 416 23
PN 92 469

Model Card Authors

cxoijve