yuashi/qwen2.5-1.5b-text2sql-merged

Qwen2.5-1.5B Text-to-SQL (LoRA fine-tune, merged)

A LoRA fine-tune of Qwen/Qwen2.5-1.5B-Instruct for text-to-SQL generation: given a database schema and a natural-language question, the model generates the corresponding SQL query. Weights in this repo are the base model with the trained LoRA adapter merged in, so it loads and runs like any standard causal LM — no PEFT wrapper required.

Model description

• Base model: Qwen2.5-1.5B-Instruct
• Fine-tuning method: QLoRA (4-bit base, LoRA rank 16, alpha 32, targeting q_proj/k_proj/v_proj/o_proj)
• Task: Text-to-SQL generation
• Training hardware: Single T4 GPU (Google Colab free tier)
• Training framework: 🤗 Transformers, PEFT, TRL (SFTTrainer)

Training data

b-mc2/sql-create-context — natural-language question, database schema (CREATE TABLE statement), and gold SQL query triples, derived from WikiSQL and Spider. Trained on a 20,000-example subset (randomly sampled, seed 42) for time budget on free-tier compute; a 95/5 train/test split was used.

Training procedure

• 1 epoch, effective batch size 16 (per-device batch 4 × gradient accumulation 4)
• Learning rate 2e-4
• Max sequence length 512 tokens
• bfloat16 compute dtype, 4-bit NF4 base quantization with double quantization
• Prompt format: Qwen's ChatML template, with a system prompt instructing SQL-only output (no explanation)

Evaluation

Evaluated on a 300-example held-out sample from the test split, using two metrics:

Metric	Score
Exact-match accuracy	73.0%
Execution validity rate	96.7%
Gold query validity (sanity ceiling)	97.7%

Exact-match accuracy: generated SQL equals gold SQL after normalizing whitespace/case.

Execution validity rate: percentage of generated queries that execute without error against an in-memory SQLite database built from the example's real schema. Note: b-mc2/sql-create-context provides schemas but no populated rows, so this measures syntactic/semantic validity against the real schema (correct table/column references, valid SQL) rather than whether results are correct — not full execution accuracy. The gold queries' own validity rate is reported as a sanity ceiling.

How to use

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "yuashi/qwen2.5-1.5b-text2sql-merged"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")

schema = "CREATE TABLE employees (id INT, name TEXT, department TEXT, salary INT)"
question = "What is the average salary in the engineering department?"

messages = [
    {"role": "system", "content": "You are a SQL expert. Given a database schema and a question, write the correct SQL query. Respond with only the SQL query, no explanation."},
    {"role": "user", "content": f"Schema:\n{schema}\n\nQuestion: {question}"},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=150, temperature=0.1, do_sample=False)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

A quantized GGUF version (Q4_K_M, ~1GB, CPU-inference friendly via llama.cpp) is also available at: https://huggingface.co/yuashi/qwen2.5-1.5b-text2sql-gguf.

Limitations

• Trained on a 20k subset of the available ~78k examples
• Execution validity is a proxy metric, not true execution accuracy (no populated rows in the training/eval data to check result correctness against)
• No adversarial or out-of-distribution schema testing performed
• Inherits any limitations and biases of the base Qwen2.5-1.5B-Instruct model

License

Apache 2.0, inherited from the base model. Training dataset (b-mc2/sql-create-context) is CC-BY-4.0.