sql-qwen2.5-1.5b-lora

QLoRA fine-tune of unsloth/Qwen2.5-1.5B-Instruct that turns English questions into SQLite queries for one fixed e-commerce schema (7 tables: customers, orders, order_items, products, categories, payments, reviews). Built to show how far a small, local model trained for one narrow task gets on 340 synthetic examples and a free GPU. Quantized GGUF for Ollama: Yash06B/sql-qwen2.5-1.5b-GGUF. Dataset: Yash06B/shop-text-to-sql. Code, data pipeline and eval harness: https://github.com/Yash06B/slm-finetune

Results

Execution accuracy = the generated SQL returns the same rows as the gold SQL on the database.

Test set: gold_test (n=50)

Model	Hardware	Exec. accuracy (95% CI)	Easy / Med / Hard	Valid SQL	p50 latency	p95 latency	Cost / 1k requests
Qwen2.5-1.5B QLoRA, Q4_K_M GGUF (ours)	RTX 4050 laptop, Ollama	62.0% (50-74)	93 / 70 / 20	86%	2.34s	2.77s	n/a
Qwen2.5-1.5B QLoRA (ours)	Colab T4 (4-bit)	56.0% (42-70)	80 / 65 / 20	88%	4.44s	9.88s	$0.483
Llama 3.2 3B (base, zero-shot)	RTX 4050 laptop, Ollama	52.0% (38-66)	80 / 65 / 7	86%	2.59s	3.29s	n/a
Qwen2.5-1.5B-Instruct (base)	Colab T4 (4-bit)	32.0% (20-44)	53 / 35 / 7	66%	3.42s	6.41s	$0.355

Test set: test_synth (n=33)

Model	Hardware	Exec. accuracy (95% CI)	Easy / Med / Hard	Valid SQL	p50 latency	p95 latency	Cost / 1k requests
Qwen2.5-1.5B QLoRA, Q4_K_M GGUF (ours)	RTX 4050 laptop, Ollama	42.4% (24-61)	67 / 40 / 33	91%	2.54s	3.03s	n/a
Qwen2.5-1.5B QLoRA (ours)	Colab T4 (4-bit)	39.4% (24-55)	67 / 47 / 17	85%	6.03s	9.75s	$0.605
Llama 3.2 3B (base, zero-shot)	RTX 4050 laptop, Ollama	15.2% (6-27)	33 / 20 / 0	94%	2.78s	3.41s	n/a
Qwen2.5-1.5B-Instruct (base)	Colab T4 (4-bit)	12.1% (3-24)	33 / 7 / 8	61%	4.08s	6.97s	$0.410

Training

model: unsloth/Qwen2.5-1.5B-Instruct
max_seq_length: 2048
lora_r: 16
lora_alpha: 32
lora_dropout: 0.0
epochs: 3.0
lr: 0.0002
batch_size: 4
grad_accum: 4
warmup_ratio: 0.03
weight_decay: 0.01
scheduler: cosine
eval_steps: 10
seed: 3407
report_to: none
run_name: None
save_merged: False
max_steps: -1
effective_batch: 16
n_train: 340
n_val: 20
family: qwen

Use with Ollama

The model expects the system prompt it was trained with (instructions + the full schema). The Modelfile in the GGUF repo bakes it in, so create the model from it rather than with ollama run hf.co/...:

hf download Yash06B/sql-qwen2.5-1.5b-GGUF --local-dir sql-qwen
cd sql-qwen
ollama create sql-qwen -f Modelfile
ollama run sql-qwen "Which 3 countries have the most customers?"

Limitations

Trained for this one schema only. It will not generalize to arbitrary databases without retraining. The training data is synthetic (written by qwen2.5-coder:7b and filtered by execution), so it inherits that model's blind spots. Large API models have not been evaluated on this test set yet.