alexwengg/shop-sql-0.6b-mlx-4bit

shop-sql-0.6b (MLX, 4-bit)

A 331 MB text-to-SQL model for one specific database: a demo online shop in SQLite. It turns questions like "top 5 products by revenue last month" into a query in about 0.2 s on a MacBook.

It's Qwen3-0.6B fine-tuned on 1,058 question/SQL pairs written and verified by Recursive Flint (40B-A2.5B), then merged and quantized to 4-bit MLX.

This is a narrow model. It knows the shop schema and answers directly without reasoning. It isn't a general text-to-SQL model and won't work on other databases.

Results

124 held-out questions, scored by execution accuracy (the SQL must return the same rows as the reference query):

Model	Accuracy	Latency p50	Latency p90	Size
Qwen3-0.6B, no fine-tuning	21.8%	0.36 s	0.64 s	1.1 GB
This model (4-bit MLX)	82.3%	0.21 s	0.32 s	331 MB
Flint 40B, 3-bit MLX, with reasoning	97.6%*	7.4 s	19.4 s	16.9 GB

By difficulty: easy 39/44, medium 44/52, hard 19/28. Latency is measured end to end with greedy decoding on an M5 Pro with 24 GB, using the mlx-lm Python API. A median answer is about 60 tokens.

* The reference answers are Flint answers that two independent Flint runs agreed on, so Flint's score is an upper bound, not an independent measurement.

The bf16 and 8-bit versions scored the same (82.3%) at 0.37 s and 0.26 s, so 4-bit loses no accuracy here.

Usage

The model expects exactly the system prompt it was trained with. That prompt holds the schema, today's date (2026-10-01) and the date conventions, and it ships in this repo as system_prompt.txt.

from huggingface_hub import hf_hub_download
from mlx_lm import generate, load

repo = "alexwengg/shop-sql-0.6b-mlx-4bit"
model, tokenizer = load(repo)
system = open(hf_hub_download(repo, "system_prompt.txt")).read()

messages = [{"role": "system", "content": system},
            {"role": "user", "content": "top 5 products by revenue last month"}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False)
print(generate(model, tokenizer, prompt, max_tokens=400))
# SELECT p.product_id, p.name, SUM(oi.quantity * oi.unit_price) AS revenue FROM order_items oi
# JOIN orders o ON oi.order_id = o.order_id JOIN products p ON oi.product_id = p.product_id
# WHERE o.status != 'cancelled' AND o.order_date >= '2026-09-01' AND o.order_date <= '2026-09-30'
# GROUP BY p.product_id, p.name ORDER BY revenue DESC LIMIT 5

Use enable_thinking=False: the model was trained to answer directly. The database (shop.db), the interactive demo and all training code are in the dataset repo, alexwengg/shop-sql-flint-distill.

How it was made

1. Database: a synthetic shop with 7 tables (customers, a two-level category tree, products, orders, order items, returns, reviews), about 41.6k rows over 2024-01 to 2026-09.
2. Teacher data: Flint, quantized to 3-bit MLX and run locally, proposed question/SQL pairs across 20 topics and 3 difficulty levels. It then answered each question again independently, with reasoning. A pair was kept only if both queries ran and returned the same non-empty result. 1,244 of 2,403 candidates (52%) passed.
3. Fine-tuning: LoRA on Qwen3-0.6B with rank 32 on all attention and MLP projections in all 28 layers, about 3 epochs (800 steps, batch 4), lr 2e-4 with cosine decay, loss on the answer only. About 14 minutes on the Mac. Validation loss fell from 1.56 to 0.10.
4. Export: the adapter merged into the base model, then quantized to 4-bit (group size 64) with mlx_lm convert. Merging alone took latency from 0.64 s to 0.37 s.

Limitations

• One database only. The schema lives in the system prompt and the model has only ever seen this shop.
• 18% of answers are wrong, mostly on hard questions (32% wrong). Typical mistakes: a missing join, a column taken from the wrong table, counting where the question asks for a sum, and reading "rate" or "ratio" questions as counts. The SQL usually still runs, so a wrong answer can look plausible.
• Read-only use. Run generated SQL against a read-only connection. The training data has no writes, but nothing stops the model from producing them.
• The reference answers are model-made. Two Flint runs agreeing is a strong filter, but nobody checked the answers by hand.
• Dates are fixed. "Today" is 2026-10-01 in both the prompt and the data.

License

Apache 2.0, the same as the base model (Qwen3-0.6B) and the teacher (Flint).