personaplex-ko-s6-text (stage6 step 1600)

ํ•œ๊ตญ์–ด ํ…์ŠคํŠธ ๋Šฅ๋ ฅ ์™„์„ฑ ์ง€์ : ํ…์ŠคํŠธ ํ…Œ์ด๋ธ” lr 2e-5(ร—20), ๋‚˜๋จธ์ง€ LoRA 1e-6. text_ppl 6,140 โ†’ 13.7. ์˜ค๋””์˜ค ๊ฒฝ๋กœ๋Š” ์‚ฌ์‹ค์ƒ ๋ฌดํ•™์Šต ์ƒํƒœ(๋ฐœํ™” ํ”„๋ ˆ์ž„ CE 4.23โ†’3.63)๋ผ ์Œ์„ฑ์€ ์˜น์•Œ์ด. ์ดํ›„ ๋ชจ๋“  ์˜ค๋””์˜ค ์Šคํ…Œ์ด์ง€(s9~)์˜ ๋ถ„๊ธฐ ๋ฒ ์ด์Šค.

๋กœ๋”ฉ

  • ํ•™์Šต/๊ฒŒ์ดํŠธ์™€ ๋™์ผํ•œ moshi ๋Ÿฐํƒ€์ž„ ๊ถŒ์žฅ: moshi.models.loaders.get_moshi_lm(model.safetensors) + tokenizer_spm_ko.model
  • HF ๋Ÿฐํƒ€์ž„: dev transformers์˜ PersonaPlexForConditionalGeneration.from_pretrained(<dir>) (config.json ํฌํ•จ๋จ). ๋‹จ, HF ๊ฒฝ๋กœ์˜ selfplay ์ƒ์„ฑ์€ ๋ฌด์Œ ์ด์Šˆ๊ฐ€ ์žˆ์Œ(๋ฆฌํฌํŠธ ยง5 ์ฐธ๊ณ ) โ€” ํ‰๊ฐ€๋Š” moshi ๋Ÿฐํƒ€์ž„ ๊ธฐ์ค€.
  • ํ…์ŠคํŠธ ์ƒ˜ํ”Œ๋ง์€ temp 0.95 / top_k 60 ๊ถŒ์žฅ (0.7/25๋Š” ๋ฐ˜๋ณต ๋ฃจํ”„ ์œ ๋ฐœ).

License / ์‚ฌ์šฉ ๊ณ ์ง€

์ด ๊ฐ€์ค‘์น˜๋Š” nvidia/personaplex-7b-v1์„ ๋ฒ ์ด์Šค๋กœ NIKL(๊ตญ๋ฆฝ๊ตญ์–ด์›) ์ผ์ƒ๋Œ€ํ™” ์ฝ”ํผ์Šค๋กœ ํŒŒ์ธํŠœ๋‹ํ•œ ํŒŒ์ƒ๋ฌผ์ž…๋‹ˆ๋‹ค. ๋ฒ ์ด์Šค ๋ชจ๋ธ์˜ ๋ผ์ด์„ ์Šค๊ฐ€ ์Šน๊ณ„๋˜๋ฉฐ, NIKL ์ฝ”ํผ์Šค ํŒŒ์ƒ๋ฌผ์ด๋ผ๋Š” ์ ์„ ๊ณ ๋ คํ•ด ์—ฐ๊ตฌ ๋ชฉ์  ์‚ฌ์šฉ์„ ๊ถŒ์žฅํ•ฉ๋‹ˆ๋‹ค. ์› ์ฝ”ํผ์Šค ๋ฐ์ดํ„ฐ ์ž์ฒด๋Š” ํฌํ•จ๋˜์–ด ์žˆ์ง€ ์•Š์Šต๋‹ˆ๋‹ค.

Downloads last month
-
Safetensors
Model size
8B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for RetentionLabs/personaplex-ko-s6-text

Finetuned
(44)
this model