roshan-soni/mnn-pocket-text2sql-full

MNN-Pocket-0.3B-Text2SQL

MNN-Pocket-0.3B-Text2SQL is a full, merged Text-to-SQL model fine-tuned from taobao-mnn/MNN-Pocket-0.3B-Base on the gretelai/synthetic_text_to_sql dataset.

This repository contains the fine-tuned model weight. It can be loaded directly with AutoModelForCausalLM.from_pretrained(...) and is intended for on-device / edge deployment via the MNN runtime after 4-bit quantization.

---

Model Details

Model type	Causal Language Model
Base model	taobao-mnn/MNN-Pocket-0.3B-Base (0.3B params)
Fine-tuning method	QLoRA on HQQ 4-bit fake-quantized base
LoRA rank / alpha	16 / 32
Trainable params during fine-tuning	7,168,000 / 65,942,912 (10.87 %)
Dataset	gretelai/synthetic_text_to_sql — 100K examples (95,000 / 5,000 train/val split)
Epochs	3
Learning rate	2e-4
Effective batch size	8 (per-device 1 × grad-accum 8)
Max sequence length	512
Precision	bf16
Target SQL dialect	SQLite (as provided by the dataset schema)

---

Intended Use

• Primary use: Generating SQL SELECT/aggregation queries from a natural-language question given an explicit schema (DDL / CREATE TABLE statements).
• Deployment target: On-device / edge inference through the MNN runtime (Android, iOS, embedded Linux).
• Prompt format: Schema is passed in a system message; the question in a user message; SQL is produced in the assistant turn.

Prompt Format

<|im_start|>system
You are a Text-to-SQL translator. Write a SQL query for the question using ONLY this schema:

CREATE TABLE employees (employee_id INT, name TEXT, age INT, department TEXT);
<|im_end|>
<|im_start|>user
What is the average age of employees in the HR department?
<|im_end|>
<|im_start|>assistant

⚠️ Always render prompts through tokenizer.apply_chat_template(messages, add_generation_prompt=True) — do not hand-write special tokens. The chat template is bundled with the model checkpoint.

---

How to Use

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_ID = "<your-username>/MNN-Pocket-0.3B-Text2SQL"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, trust_remote_code=True)
if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token

model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    torch_dtype=torch.bfloat16 if torch.cuda.is_bf16_supported() else torch.float16,
    device_map="cuda",
    trust_remote_code=True,
)
model.eval()

schema = "CREATE TABLE employees (employee_id INT, name TEXT, age INT, department TEXT);"
question = "What is the average age of employees in the HR department?"

messages = [
    {
        "role": "system",
        "content": (
            "You are a Text-to-SQL translator. "
            "Write a SQL query for the question using ONLY this schema:\n\n"
            f"{schema}"
        ),
    },
    {"role": "user", "content": question},
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    out = model.generate(
        **inputs,
        max_new_tokens=256,
        num_beams=4,
        early_stopping=True,
        no_repeat_ngram_size=3,
        do_sample=False,
        pad_token_id=tokenizer.eos_token_id,
        eos_token_id=tokenizer.eos_token_id,
    )

sql = tokenizer.decode(
    out[0][inputs["input_ids"].shape[1]:],
    skip_special_tokens=True,
).strip()

print(sql)
# SELECT AVG(age) FROM employees WHERE department = 'HR';

On-device (MNN)

Export the merged fine-tuned model to MNN with 4-bit quantization:

python transformers/llm/export/llmexport.py \
    --path <your-username>/MNN-Pocket-0.3B-Text2SQL \
    --export mnn \
    --quant_bit 4 \
    --quant_block 64 \
    --hqq

---

Training Data

gretelai/synthetic_text_to_sql — 100,000 synthetic text↔SQL pairs across 100+ domains, LLM-verified for SQL correctness and standards compliance. Each example provides a sql_prompt (natural-language question), sql_context (DDL + sample inserts), and the gold sql query.

A 95/5 split was used (95,000 train / 5,000 validation).

---

Evaluation

Status: pending. Fill in after running execution-accuracy evaluation against the 5,000-example held-out split. Report execution accuracy (does the generated SQL run and return the correct rows?) — not BLEU/ROUGE.

Decoding Execution Accuracy Notes Greedy (num_beams=1) TBD fastest; matches typical MNN on-device path Beam-4 TBD Beam-8 TBD

To reproduce:

python transformers/llm/eval/eval_lora_effect.py \
    --base_model <your-username>/MNN-Pocket-0.3B-Text2SQL \
    --eval_data  /path/to/eval_text2sql

---

Limitations

· Small model → limited reasoning. 0.3B parameters struggles with nested subqueries, multi-CTE queries, and complex window functions. Accuracy drops sharply as query complexity rises beyond 3–4 joins. · Schema must be explicit. The model does not retrieve schema — you must supply the relevant CREATE TABLE statements in the system prompt. Performance degrades with very large schemas that exceed the 512-token context. · Synthetic training data. gretelai/synthetic_text_to_sql is fully synthetic — the model may not generalize perfectly to messy, real-world enterprise schemas (inconsistent naming, legacy structures, dialect-specific quirks). · SQLite dialect bias. The dataset skews toward SQLite-flavored SQL. Dialect-specific syntax (PostgreSQL LATERAL, BigQuery QUALIFY, etc.) is not reliably produced. · Not safe for unsupervised execution. Generated SQL should be parsed, sandboxed, and validated before running against any production database. It has not been trained to avoid destructive statements, and it may hallucinate tables/columns absent from the provided schema. · QLoRA-trained base. Fine-tuning used an HQQ 4-bit fake-quantized base. The merged bf16 weights are close to, but not bit-identical with, the exact training-time fake-quantized distribution. For strict reproduction, use the MNN evaluation path.

---

Bias, Risks, and Ethical Considerations

· Trained only on English natural-language questions. Non-English input is out of distribution. · The synthetic dataset may under-represent schemas from non-Western business domains. Query accuracy may vary systematically across domains. · Because the model emits executable code, prompt injection through the schema or question is a real risk — a malicious schema string could coax the model into emitting DROP/DELETE statements. Always sanitize and constrain execution. · No PII-specific filtering was applied during training. Do not train further on or expose PII-containing databases without appropriate controls.

---

Citation

@misc{mnn_pocket_text2sql_2025,
  title  = {MNN-Pocket-0.3B-Text2SQL},
  author = {<your name>},
  year   = {2025},
  note   = {Full merged QLoRA fine-tune of taobao-mnn/MNN-Pocket-0.3B-Base on gretelai/synthetic_text_to_sql},
  url    = {https://huggingface.co/<your-username>/MNN-Pocket-0.3B-Text2SQL}
}

@misc{gretelai_synthetic_text_to_sql,
  title  = {Synthetic Text-to-SQL},
  author = {Gretel.ai},
  year   = {2024},
  url    = {https://huggingface.co/datasets/gretelai/synthetic_text_to_sql}
}

---

Acknowledgements

· Base model: taobao-mnn/MNN-Pocket-0.3B-Base · Dataset: gretelai/synthetic_text_to_sql · Framework: MNN (mnn_qlora.py, llmexport.py)