qwen0.5b-alpaca-lora
LoRA adapter for Qwen/Qwen2.5-0.5B fine-tuned on 5,000 samples from
yahma/alpaca-cleaned for 1 epoch. Trained with trl.SFTTrainer using
LoRA rank=8, alpha=16, targeting q_proj, k_proj, v_proj, o_proj.
Final training loss: 1.382, token accuracy: 65.6%.
Usage
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B")
model = PeftModel.from_pretrained(base, "zc88/qwen0.5b-alpaca-lora")
tokenizer = AutoTokenizer.from_pretrained("zc88/qwen0.5b-alpaca-lora")
Part of llm-bootcamp Day 7.
- Downloads last month
- 4
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for zc88/qwen0.5b-alpaca-lora
Base model
Qwen/Qwen2.5-0.5B