EuricoGVP/Qwen3.5-4b_text2sql

Qwen3.5-4B Text-to-SQL

This repository contains the full model weights and configuration files in the Hugging Face Transformers format. The LoRA adapter is merged into the weights, so PEFT is not needed.

These artifacts are compatible with Hugging Face Transformers and vLLM.

Qwen3.5-4B Text-to-SQL is a fine-tune of Qwen/Qwen3.5-4B that generates one SQLite query from a database schema, an optional hint and a question.

Qwen3.5-4B Text-to-SQL Highlights

Compared with the base model, this fine-tune delivers:

• Higher Execution Accuracy: 54.76% on BIRD dev with the plain schema, 56.19% with column descriptions and 81.82% on Spider dev, between 4.7 and 9.9 percentage points above the base model.
• Significant Gain in All Runs: exact McNemar p below 1e-04 in all three runs, with higher accuracy at every difficulty level of both benchmarks.
• Fewer Execution Errors on BIRD: from 246 to 80 on BIRD plain and from 223 to 76 on BIRD desc. On Spider dev they went from 26 to 28.

For more details, please refer to the notebooks.

Model Overview

• Type: Causal Language Model with Vision Encoder
• Task: Text-to-SQL for SQLite
• Base Model: Qwen/Qwen3.5-4B
• Language Model
  • Number of Parameters: 4B
  • Hidden Dimension: 2560
  • Number of Layers: 32
  • Hidden Layout: 8 × (3 × (Gated DeltaNet → FFN) → 1 × (Gated Attention → FFN))
• Fine-Tuning
  • Method: LoRA with Unsloth, merged into the weights
  • Rank and Alpha: 16 and 16
  • Epochs: 1
  • Precision: bf16
  • Scope: language model only, the vision encoder was not changed
• Max Training Length: 4,096 tokens
• Evaluated Context Length: 8,192 tokens

Training Data

Source	Examples	License
EuricoGVP/birdsql_complete_trainset (BIRD train)	5,980	CC BY-SA 4.0
EuricoGVP/spider_sql_complete (Spider train), without formula_1, which is also in BIRD dev	8,097	CC BY-SA 4.0
seeklhy/SynSQL-2.5M subset, with long context and names that need quotes	2,105	Apache-2.0
llmsql-bench/llmsql-benchmark subset, with columns that need quotes	950	MIT
Total	17,132

Model Family

Model	Base Model	Weights
Qwen3.5-4B Text-to-SQL (this repository)	Qwen/Qwen3.5-4B	about 9 GB
Qwen3.5-9B Text-to-SQL	Qwen/Qwen3.5-9B	about 19 GB

Benchmark Results

Execution accuracy (EX) in percent, against Qwen/Qwen3.5-4B with the same prompts. BIRD dev has 1,534 questions on 11 databases, with two variants: plain (schema only) and desc (schema with column descriptions). Spider dev has 1,034 questions on 20 databases.

Benchmark	Base	This Model	Only This Model Correct	Only Base Correct	McNemar p
BIRD dev, plain	46.22	54.76	230	99	3.75e-13
BIRD dev, desc	46.28	56.19	245	93	6.34e-17
Spider dev	77.08	81.82	95	46	4.49e-05

BIRD (dev)

Metric	Base, plain	This Model, plain	Base, desc	This Model, desc
Simple	54.49	61.08	53.19	62.27
Moderate	35.56	47.20	37.93	48.28
Challenging	27.59	38.62	28.97	42.76
Execution errors	246	80	223	76

Spider (dev)

Metric	Base	This Model
Easy	90.73	92.34
Medium	82.96	87.22
Hard	67.82	74.14
Extra	50.60	59.64
Execution errors	26	28

Model Family Comparison

Model	BIRD plain	BIRD desc	Spider dev
Qwen3.5-4B base	46.22*	46.28*	77.08
Qwen3.5-4B fine-tuned	54.76	56.19	81.82
Qwen3.5-9B base	51.11	52.09	77.27
Qwen3.5-9B fine-tuned	57.17	59.32	82.01

Values from EuricoGVP/Qwen3.5-4b_text2sql and EuricoGVP/Qwen3.5-9b_text2sql. *Measured with the same benchmark notebook as the fine-tuned models.

A query counts as correct when it returns the same result as the gold query on the real database. BIRD compares results as sets, as in the official evaluation.py. Spider follows exec_eval.py from the official test-suite, but runs only on the original database, without the extra test-suite databases, so the official score would be equal or lower.

Quickstart

The model was trained and evaluated in non-thinking mode, with the prompt below. Keep enable_thinking=False and this format to reproduce the results.

Prompt Template

Replace None with the hint when you have one. The model answers with a single SQL code block.

SYSTEM_PROMPT = """You are an expert SQLite data analyst.

You will be given a database schema, an optional hint, and a question.
Write a single SQLite query that answers the question.

Rules:
- Use only the tables and columns present in the schema.
- Use the hint when it is provided; it explains domain terms in the question.
- Return exactly the columns the question asks for, nothing more.
- Output only the query, inside a single ```sql code block.
- Do not explain your reasoning and do not write any text outside the code block."""

schema = """CREATE TABLE singer (Singer_ID int PRIMARY KEY, Name text);
CREATE TABLE song (Song_ID int PRIMARY KEY, Title text, Singer_ID int);"""
question = "Show the names of singers that have more than one song."

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": f"Database schema:\n{schema}\n\nHint: None\n\nQuestion: {question}"},
]

Inference with vLLM

# pip install vllm
from vllm import LLM, SamplingParams

llm = LLM(model="EuricoGVP/Qwen3.5-4b_text2sql", max_model_len=8192, language_model_only=True)
sampling = SamplingParams(temperature=0, max_tokens=2048)

outputs = llm.chat([messages], sampling, chat_template_kwargs={"enable_thinking": False})
print(outputs[0].outputs[0].text)

Inference with Transformers

# pip install "transformers>=5.22.1" accelerate
from transformers import AutoModelForImageTextToText, AutoTokenizer

model_id = "EuricoGVP/Qwen3.5-4b_text2sql"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(model_id, dtype="auto", device_map="auto")

inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, enable_thinking=False, return_tensors="pt", return_dict=True
).to(model.device)
output = model.generate(**inputs, max_new_tokens=2048, do_sample=False, eos_token_id=tokenizer.eos_token_id)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Best Practices

To achieve the benchmark performance, we recommend the following settings:

1. Prompt Format: use the system prompt and the user message above without changes. Other formats were not evaluated.
2. Greedy Decoding: use temperature=0, as in the benchmarks.
3. Column Descriptions: include them in the schema when you have them. On BIRD dev they raised EX from 54.76% to 56.19%, although this difference alone is not statistically significant (exact McNemar p = 0.08).
4. Read-Only Execution: run the generated queries on a read-only connection and review the results. The model does not run or validate queries.

Limitations

• The model generates SQLite only. Both benchmarks are in English and use SQLite, so other languages and dialects were not evaluated.
• Most wrong answers run without error: 614 of 694 on BIRD plain, 596 of 672 on BIRD desc and 160 of 188 on Spider dev.
• The most common execution error is no such column: 65 of 80 on BIRD plain, 64 of 76 on BIRD desc and 25 of 28 on Spider dev.
• Examples over 4,096 tokens were not used in training, so behavior on very large schemas is not well measured.

License

• Model weights: Apache-2.0, the same license as the base model. This is a modified version of Qwen/Qwen3.5-4B: the weights were changed by LoRA fine-tuning.
• Files in data/: CC BY-SA 4.0, since they contain questions, schemas and queries from BIRD and Spider, with attribution to the original authors. The excerpts from SynSQL-2.5M (Apache-2.0) and LLMSQL (MIT, derived from WikiSQL, BSD-3-Clause) are compatible with this redistribution. Anyone using these data must respect the original license of each dataset.
• Notebooks: Apache-2.0.

Citation

If you find this work helpful, feel free to cite it.

@misc{pereira2026qwen35text2sql4b,
  title  = {{Qwen3.5-4B Text-to-SQL}},
  author = {Eurico Pereira},
  year   = {2026},
  url    = {https://huggingface.co/EuricoGVP/Qwen3.5-4b_text2sql}
}

Base model and datasets

@misc{qwen3.5,
    title  = {{Qwen3.5}: Towards Native Multimodal Agents},
    author = {{Qwen Team}},
    month  = {February},
    year   = {2026},
    url    = {https://qwen.ai/blog?id=qwen3.5}
}

@article{li2024can,
  title={Can llm already serve as a database interface? a big bench for large-scale database grounded text-to-sqls},
  author={Li, Jinyang and Hui, Binyuan and Qu, Ge and Yang, Jiaxi and Li, Binhua and Li, Bowen and Wang, Bailin and Qin, Bowen and Geng, Ruiying and Huo, Nan and others},
  journal={Advances in Neural Information Processing Systems},
  volume={36},
  year={2024}
}

@inproceedings{Yu&al.18c,
  title     = {Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task},
  author    = {Tao Yu and Rui Zhang and Kai Yang and Michihiro Yasunaga and Dongxu Wang and Zifan Li and James Ma and Irene Li and Qingning Yao and Shanelle Roman and Zilin Zhang and Dragomir Radev},
  booktitle = "Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing",
  address   = "Brussels, Belgium",
  publisher = "Association for Computational Linguistics",
  year      = 2018
}

@inproceedings{llmsql_bench,
  title={LLMSQL: Upgrading WikiSQL for the LLM Era of Text-to-SQL},
  author={Pihulski, Dzmitry and Charchut, Karol and Novogrodskaia, Viktoria and Koco{\'n}, Jan},
  booktitle={2025 IEEE International Conference on Data Mining Workshops (ICDMW)},
  year={2025},
  organization={IEEE}
}

Acknowledgements

Thanks to the Qwen Team, Unsloth, vLLM and the authors of BIRD, Spider, SynSQL-2.5M, LLMSQL and WikiSQL.