BharatJitendra/intentgrasp-qlora-adapter

Model Card: IntentGrasp QLoRA Adapter (Qwen2.5-1.5B)

A LoRA adapter that finetunes Qwen2.5-1.5B-Instruct to classify speaker intent and output structured JSON.

Model Details

• Developed by: [your name / handle]
• Model type: LoRA adapter for causal LM (intent classification → structured JSON)
• Language(s): English
• License: Derived from IntentGrasp (CC-BY-NC-SA 4.0) — non-commercial, research/learning use only
• Finetuned from: Qwen/Qwen2.5-1.5B-Instruct

Model Sources

• Repository: [your GitHub repo URL]
• Dataset: yuweiyin/IntentGrasp

Uses

Direct Use

Given a context (query, dialogue, or monologue), a question, and a list of candidate intents, the model selects the correct intent(s) and returns them as JSON, e.g. {"answer": ["7"], "intent": ["To book flights..."]}.

Out-of-Scope Use

Not for production or commercial use (non-commercial dataset license). Trained on a narrow multiple-choice intent format; not a general-purpose assistant.

Limitations

• Accuracy is strongly distribution-dependent: 90.5% on in-distribution validation but 37.9% on the harder balanced gem split.
• Errors are dominated by over-prediction on ambiguous multi-intent cases.

How to Get Started

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
model = PeftModel.from_pretrained(base, "[your-username]/intentgrasp-qlora-adapter")

Training Details

• Training data: IntentGrasp (~249k rows after filtering invalid-label rows)
• Method: QLoRA — 4-bit (nf4) base + rank-16 LoRA adapter on all linear layers (~1.2% params trainable)
• Hyperparameters: 1 epoch, LR 2e-4 (cosine), effective batch [BATCH], max_length 1024, assistant-only loss masking

Evaluation

Split	JSON validity	Accuracy
Validation (in-distribution)	100%	90.5%
gem (harder, balanced)	100%	37.9%

Base model (untrained), for comparison: 0.5% schema conformance, 41.5% lenient accuracy.

Summary: Finetuning made JSON output 100% reliable and roughly doubled in-distribution accuracy, but generalization to the harder split is partial — format-following transfers completely while task accuracy is distribution-dependent.

Hardware

• Single GPU (QLoRA / 4-bit), ~1 epoch on ~249k examples.