fg-sql-merged

A fine-tuned text-to-SQL model for the fg_organization HR/ERP database schema. Given a natural-language question, it generates the corresponding MySQL query. This repo is the merged model (base weights + LoRA adapter folded together, full fp16 precision) — for the standalone adapter alone, see yoonboom/fg-sql-lora-adapter.

Model Details

• Base model: Qwen/Qwen2.5-Coder-7B-Instruct
• Fine-tuning method: QLoRA (4-bit base + LoRA adapter during training, merged back to fp16 for this repo)
• LoRA config: r=16, alpha=32, dropout=0.05, target modules q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Precision: fp16 (merged); a q8_0 GGUF quantization is also available for llama.cpp/Ollama serving at yoonboom/fg-sql-gguf
• Language: English (natural-language input), MySQL 8.0 dialect (SQL output)

Intended Use

Translate natural-language questions about a fictional company's HR/ERP data into MySQL queries against the fg_organization schema (18 tables covering organization structure, workforce, projects/tasks, and finance). Intended to run behind a guardrail layer that validates generated SQL against the real schema before execution (see sql_schema_guard.py in the companion repo) — this model should not be trusted to execute SQL directly against a production database without that validation step.

Out of Scope

• Any other database schema — this model is specifically tuned to fg_organization's tables/columns/ID conventions and will not generalize to arbitrary SQL schemas.
• Recruitment/candidate data, IT asset tracking, employee training records, and real external client identity — the underlying schema has no tables for any of these; the model is trained to decline these rather than fabricate a table.
• True cost/profit/margin calculations — the schema has no cost/rate column anywhere, only revenue and hours logged as separate figures.

Training Data

• 400 hand-written/generated question→SQL pairs (320 train / 40 val / 40 test), stratified across ~10-11 categories: simple lookups, joins, aggregations, date/time filtering, manager-hierarchy recursion, and mixed-complexity queries (CTEs, EXISTS, correlated subqueries).
• Schema: fg_organization, 18 tables, MySQL 8.0 (WITH RECURSIVE support required for hierarchy queries).

Training Procedure

• Hardware: Google Colab free-tier, Tesla T4 GPU (16GB VRAM, fp16 only — no bf16 support)
• Method: QLoRA — 4-bit quantized base (bitsandbytes, nf4, double-quant) + LoRA adapter (peft), trained via transformers.Trainer
• Hyperparameters: 3 epochs, effective batch size 16 (per-device batch 1 × grad accumulation 16), learning rate 2e-4, cosine schedule, 3% warmup, paged_adamw_8bit optimizer
• Loss masking: assistant-turn-only (system + user prompt tokens masked to -100, loss computed only on the generated SQL)

Training Results

Step	Train loss	Val loss
10	0.369	0.227
20	0.195	0.155
30	0.102	0.124
40	0.075	0.113
50	0.065	0.110
60	0.048	0.109

Val loss plateaus over the final 20 steps while train loss keeps falling — 3 epochs was about the right stopping point for this dataset size.

Evaluation

• Exact-match on held-out test set (40 examples): 30% strict string-match; substantially higher (68%) when accounting for functionally-equivalent-but-differently-styled SQL (renamed aliases, different but valid ORDER BY choices, extra helpful columns).
• Known weak categories (as of this checkpoint): resolving named roles/titles without hardcoded lookups, descendant-direction recursive hierarchy questions ("everyone under X" vs. "who X reports to"), and a tendency toward schema hallucination under harder aggregation/CTE questions (invented column/ID names that don't exist in the real schema).
• A static schema-fidelity checker (regex/alias-aware, no DB required) is used alongside this model in production to catch hallucinated tables/columns/IDs before execution.

Limitations

• Trained on a relatively small dataset (400 pairs) for a 7B model — some brittleness on question phrasings not well-represented in training data should be expected, particularly for hierarchy-direction and role/title resolution questions (see Evaluation above).
• This model can still generate SQL referencing non-existent columns/tables/IDs when uncertain, rather than reliably declining — always validate output against the real schema before execution.
• Specific to the fg_organization schema; not a general-purpose text-to-SQL model.

License

Apache 2.0 (inherited from the Qwen2.5-Coder-7B-Instruct base model).