qwen2.5-coder-3b-text2sql-qlora

QLoRA adapter for Qwen/Qwen2.5-Coder-3B-Instruct, fine-tuned to turn a table schema and a question into a SQL query.

Code and full evaluation: https://github.com/VarshithaChappidi/text2sql-qlora

Results (300 held-out examples)

Model	Strict exact match	Normalized exact match
Base, prompted	0.023	0.570
This adapter	0.767	0.773

Normalized match unifies quote style, drops wrapping quotes, treats INNER JOIN as JOIN, and ignores AND order. It still rejects some valid SQL written differently, so it is a lower bound.

Usage

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_id = "Qwen/Qwen2.5-Coder-3B-Instruct"
tok = AutoTokenizer.from_pretrained(base_id)
base = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype=torch.float16, device_map="auto")
model = PeftModel.from_pretrained(base, "Varshitha2603/qwen2.5-coder-3b-text2sql-qlora")

schema = "CREATE TABLE singer (name VARCHAR, age INT)"
question = "How many singers are older than 30?"
msgs = [
    {"role": "system", "content": "You are a SQL expert. Given a table schema and a question, reply with only the SQL query."},
    {"role": "user", "content": f"Schema:\n{schema}\n\nQuestion: {question}"},
]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
inputs = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=128, do_sample=False)
print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

Training

• Data: 5,000 examples from b-mc2/sql-create-context (300 other examples held out for testing)
• QLoRA: 4-bit NF4, rank 16, alpha 32, dropout 0.05, all attention and MLP projections (29.9M trainable parameters)
• 1 epoch, learning rate 2e-4 with cosine schedule, effective batch size 16, loss on answer tokens only
• Free Colab T4 GPU, about 21 minutes. Loss fell from 0.212 (step 25) to 0.046 (step 300).

Limitations

• Evaluated by exact match, not execution, because the dataset has no table contents.
• Trained and tested only on single-table-style questions from one dataset, plus a few Spider-style joins. Multi-table joins are a known weak spot.
• Gold labels contain ambiguity and typos. Single training run, no variance estimate.
• The adapter inherits the license terms of the base model and the dataset. Check both before commercial use.