---
language:
- en
license: apache-2.0
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- trl
base_model: unsloth/llama-3-8b-Instruct-bnb-4bit
pipeline_tag: text-generation
datasets:
- LimYeri/LeetCode_Python_Solutions
---

# Uploaded  model

- **Developed by:** LimYeri
- **License:** apache-2.0
- **Finetuned from model :** unsloth/llama-3-8b-Instruct-bnb-4bit

This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)

### Training Setting
The following hyperparameters are used during SFT:
- num_epochs: 3
- learning_rate: 1e-4
- max_seq_length: None
- optimizer: adamw_8bit
- lr_scheduler_type: linear
- warmup_ratio: 0.03
- weight_decay: 0.01
- lora_rank: 16
- lora_alpha: 16
- lora_dropout: 0
- gradient_checkpointing: true
- fp16: not is_bfloat16_supported()
- bf16: is_bfloat16_supported()