Uploaded model
- Developed by: LimYeri
- License: apache-2.0
- Finetuned from model : unsloth/llama-3-8b-Instruct-bnb-4bit
This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.
Training Setting
The following hyperparameters are used during SFT:
- num_epochs: 1
- learning_rate: 2e-4
- max_seq_length: None
- optimizer: adamw_8bit
- lr_scheduler_type: linear
- warmup_steps: 5
- max_steps: 0
- lora_rank: 16
- lora_alpha: 16
- lora_dropout: 0
- gradient_checkpointing: true
- fp16: not is_bfloat16_supported()
- bf16: is_bfloat16_supported()
- Downloads last month
- 3