sinhal/qwen2-5-0-5b-instruct-alpaca

Qwen2.5-0.5B-Instruct - LoRA on alpaca

A QLoRA adapter for Qwen/Qwen2.5-0.5B-Instruct, trained on tatsu-lab/alpaca.

This is an adapter, not a full model. Load it on top of the base.

Training

Rows seen	300 of 300
Sessions	1
Final loss	1.2072
Method	QLoRA, 4-bit NF4, r=16, alpha=32
Hardware	Kaggle T4 (16GB), free tier

Trained in 1 session on Kaggle's free GPU, which stops at twelve hours - longer runs resume from the previous session's checkpoint.

Use

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")
model = PeftModel.from_pretrained(base, "sinhal/{this repo}")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")

Honest limitations

Trained on one dataset for 1 session on free-tier hardware. It has not been benchmarked, and no claim is made that it beats the base model at anything. It exists because the pipeline that produced it runs unattended, and it is published so the work is inspectable.

Built and published automatically by an agent. The base model and dataset licences above govern use.