swathikchhuggingface/ag-news-lora

AG News — lora

Part of lora-methods-comparison — a rigorous, apples-to-apples comparison of Full fine-tuning, LoRA, LoRA-FA, QLoRA, and TinyLoRA, trained on the same task (AG News topic classification), same base model (Qwen/Qwen2.5-0.5B-Instruct), same GPU, same evaluation harness.

This checkpoint

Standard LoRA (peft.LoraConfig, rank 16, alpha 32) — low-rank adapters on attention and MLP projections, base weights frozen.

Trainable parameters: 8,798,208 Test accuracy: 85.5% Peak GPU memory during training: 5.17074176 GB Training wall-clock time: 105.36816549301147s

How to load

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")
model = PeftModel.from_pretrained(base, "swathikchhuggingface/ag-news-lora")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")

See the repo README for the full comparison across all 5 methods, including methodology and the complete results table.