jk200201/qwen2.5-coder-7b-bird-cot-GGUF

Qwen2.5-Coder-7B BIRD CoT (Text-to-SQL) GGUF

GGUF quantizations of jk200201/qwen2.5-coder-7b-bird-cot, a 7B text-to-SQL model that reasons step by step over a database schema and then writes the SQL. Runs locally on CPU or a single consumer GPU with llama.cpp or Ollama.

On BIRD dev (realistic, messy schemas, the harder text-to-SQL benchmark) the full-precision model reaches 52.1% greedy result accuracy and 58.5% with self-consistency (Best-of-N, K=8). With self-consistency it matches DeepSeek V4-Pro (1.6T parameters) at roughly 0.4% of the size, running locally.

Files

File	Quant	Size	Notes
qwen2.5-coder-7b-bird-cot-Q4_K_M.gguf	Q4_K_M	4.4 GB	Recommended. Best size/quality trade-off.
qwen2.5-coder-7b-bird-cot-Q5_K_M.gguf	Q5_K_M	5.1 GB	Higher fidelity, slightly larger.
qwen2.5-coder-7b-bird-cot-Q8_0.gguf	Q8_0	7.6 GB	Near lossless.
qwen2.5-coder-7b-bird-cot-f16.gguf	F16	15 GB	Full precision reference.

Start with Q4_K_M. Move up to Q5_K_M or Q8_0 if you have the memory and want to close the small accuracy gap.

Quick start (Ollama)

# Download the Q4_K_M file, then from the same directory:
ollama create bird-cot -f Modelfile
ollama run bird-cot

Modelfile:

FROM ./qwen2.5-coder-7b-bird-cot-Q4_K_M.gguf

TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
{{ .Response }}<|im_end|>
"""

SYSTEM """You are an expert SQLite query writer. Reason step by step about the schema and the question, then output the final query in a fenced sql code block."""

PARAMETER temperature 0
PARAMETER num_ctx 8192
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"

Then send a schema and a question:

Given the database schema and question, work out the correct SQLite query step by step.

Database Schema:
CREATE TABLE singer (Singer_ID INT, Name TEXT, Age INT);

Question: How many singers are older than 40?

Think step by step, then give the final answer in a fenced sql block.

The model returns its reasoning followed by a fenced SQL block. Take the last SQL block as the query.

Quick start (llama.cpp)

./llama-cli -m qwen2.5-coder-7b-bird-cot-Q4_K_M.gguf -no-cnv --temp 0 -n 512 \
  -p $'<|im_start|>system\nYou are an expert SQLite query writer. Reason step by step about the schema and the question, then output the final query in a fenced sql code block.<|im_end|>\n<|im_start|>user\nGiven the database schema and question, work out the correct SQLite query step by step.\n\nDatabase Schema:\nCREATE TABLE singer (Singer_ID INT, Name TEXT, Age INT);\n\nQuestion: How many singers are older than 40?\n\nThink step by step, then give the final answer in a fenced sql block.<|im_end|>\n<|im_start|>assistant\n'

Results: BIRD dev (execution result accuracy)

All models use a single chain-of-thought sample unless noted, so the comparison is like for like.

Model	Params	Result accuracy
Base Qwen2.5-Coder-7B-Instruct	7B	27.0%
This model, greedy	7B	52.1%
This model, self-consistency (K=8)	7B	58.5%
DeepSeek V4-Pro	1.6T	58.7%
GLM 5.2	744B	63.0%

Result accuracy is the fraction of queries whose SQL executes to the same rows as the gold query (BIRD's official execution metric). The reported figures are for the full-precision model. Quantization retains most of that accuracy; Q4_K_M is the recommended balance.

Training

Reasoning distillation (CoT-SFT). A strong teacher (Qwen3-Coder-480B) generated step-by-step CoT solutions on BIRD train. Only execution-verified-correct chains were kept (5,593 examples), then supervised fine-tuned into the 7B. Distilling the teacher's reasoning generalized across BIRD's cross-domain dev databases better than distilling SQL answers directly.

Config: QLoRA (4-bit NF4, LoRA rank 32, alpha 64), 2 epochs, learning rate 2e-4 cosine, max sequence length 8192.

Model family

Artifact	Repo
GGUF (this repo)	jk200201/qwen2.5-coder-7b-bird-cot-GGUF
Merged model	jk200201/qwen2.5-coder-7b-bird-cot
LoRA adapter	jk200201/qwen2.5-coder-7b-bird-cot-lora
Training data	jk200201/bird-cot-sft

Limitations

• Tuned for BIRD-style analytic SQL over realistic schemas. Unusual dialects or domains may need adaptation. It emits SQLite dialect.
• Greedy (52.1%) is the deployable single-shot number. The 58.5% figure needs K=8 self-consistency (roughly 8x inference).
• This is a 7B model. Review generated SQL before running it on production data.

Acknowledgements

Base model: Qwen2.5-Coder by Alibaba Qwen. Teacher: Qwen3-Coder-480B. Benchmark: BIRD (bird-bench.github.io).