metadata
library_name: transformers
datasets:
- kyujinpy/OpenOrca-KO
pipeline_tag: text-generation
Llama-3-Ko-OpenOrca
Model Details
Model Description
Original model: beomi/Llama-3-Open-Ko-8B
Dataset: kyujinpy/OpenOrca-KO
Training details
Training: Axolotl을 이용해 LoRA-8bit로 4epoch 학습 시켰습니다.
- sequence_len: 4096
- bf16
학습 시간: A6000x2, 6시간