m4xx101/llama-3.2-3b-lora

Llama-3.2-3B LoRA — OpenHermes 2.5 Fine-Tune

A QLoRA adapter fine-tuning Meta's Llama 3.2 3B Instruct on OpenHermes 2.5 — one of the largest and most diverse instruction-tuning datasets available. This adapter punches well above its 97 MB weight class: load it alongside the 4-bit base model and you get a compact, fast, and surprisingly capable chat model that runs comfortably on a single consumer GPU or even CPU with quantization.

Why this model

• Runs on anything. 3B parameters at 4-bit + a 97 MB LoRA adapter. Fits on a laptop GPU, a free Colab T4, or even CPU-only with llama.cpp.
• OpenHermes pedigree. Trained on teknium/OpenHermes-2.5 — the gold-standard open instruction dataset spanning chat, reasoning, code, roleplay, and general knowledge. Not a narrow vertical fine-tune; this is a broad capability upgrade.
• QLoRA, not a full fine-tune. 16-rank LoRA on all attention + MLP projections (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj). Efficient, mergeable, and trivial to stack with other adapters.
• Hello-world project, real results. 30 training steps on a single GPU — proof that you don't need a datacenter to meaningfully improve an open model.

Quick Start

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "m4xx101/llama-3.2-3b-lora"
base_model = "unsloth/Llama-3.2-3B-Instruct-bnb-4bit"

model = AutoModelForCausalLM.from_pretrained(
    base_model,
    load_in_4bit=True,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(model, model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id)

messages = [
    {"role": "system", "content": "You are a helpful AI assistant."},
    {"role": "user", "content": "Explain quantum computing in one paragraph."},
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to("cuda")
outputs = model.generate(inputs, max_new_tokens=256, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Merge into a full model

merged = model.merge_and_unload()
merged.save_pretrained("./llama-3.2-3b-openhermes-merged")
tokenizer.save_pretrained("./llama-3.2-3b-openhermes-merged")

Training Details

Parameter	Value
Base model	unsloth/Llama-3.2-3B-Instruct (4-bit BnB)
Dataset	teknium/OpenHermes-2.5
Method	QLoRA (4-bit NF4 + LoRA)
LoRA rank (r)	16
LoRA alpha	16
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Training steps	30
Final loss	0.72
Framework	Unsloth + PEFT 0.18.1

Limitations

• This is a LoRA adapter — not a standalone model. You MUST load it alongside the base model.
• 30 training steps means the adapter captures broad patterns from OpenHermes but hasn't converged deeply. Expect improved conversational tone and instruction-following, not specialized expertise.
• The base model is quantized (bnb-4bit). Merging produces a 4-bit full model; exporting to fp16/bf16 requires dequantization.
• OpenHermes 2.5 contains diverse data including roleplay and creative writing — the model may produce unexpected outputs in those domains.

License

Apache 2.0 — same as the base Llama 3.2 model. Use it, merge it, stack it, ship it.

---

Built with Unsloth — the fastest way to fine-tune open LLMs on consumer hardware.