cwolff/qwen3.5-2b-grpo-bird

Qwen3.5-2B · agentic GRPO on BIRD

A 2B text-to-SQL agent, not a single-shot SQL generator. Starting from the stock Qwen/Qwen3.5-2B (no supervised warm start), it was trained with GRPO-style RL for 1350 steps against real SQLite execution: it explores a database with tools, then submits a query, and is rewarded on whether the result matches the gold answer.

Training data: BIRD train (simple + moderate). Related models, identical recipe on different training data: cwolff/qwen3.5-2b-grpo-sqale (SQaLe3) and cwolff/qwen3.5-2b-grpo-synsql (SynSQL).

Results — BIRD dev

Measured on this repo as published, loaded straight into vLLM. n=300 BIRD dev rows (seeded), 6-turn agentic episodes, T=0.6 / top-p 0.95 / top-k 20, scored by executing against the real SQLite databases. execution_accuracy_official is BIRD-leaderboard set semantics; strict is order- and duplicate-sensitive. full puts the DDL in the prompt; withheld gives the model only the question and makes it discover the schema through tools.

	stock Qwen3.5-2B	this model	stock Qwen3.8-27B
BIRD full, official	0.290	0.553	0.687
BIRD full, strict	—	0.513	—
BIRD withheld, official	0.193	0.547	0.693
BIRD withheld, strict	—	0.510	—

All five bars were produced by the same harness in the same way, which is what makes them comparable. Two things the chart says out loud:

• RL on a 2B works, and works hardest where the schema is hidden. Against its own starting point the gain is +0.263 with the schema in the prompt and +0.354 with it withheld -- exploration is where the training pays.
• A 13x larger untrained model still wins. Stock Qwen3.8-27B reaches 0.687 / 0.693 without any training on this task. If you have the VRAM for a 27B and do not need a 2B's latency or cost, it is the better model; these checkpoints are for the case where you need the small one.

Episode behaviour: submit rate 0.60 / 0.59 (full/withheld), generated SQL executes 97.7% / 97.0% of the time, 4.1 / 5.1 mean turns.

On the stock-2B column. It is a fresh run of this harness against Qwen/Qwen3.5-2B, not the figure the trainer logged at step 0 (which was 0.13-0.16 on full). The two disagree, and the cause is not settled: the behavioural counters match closely but the fallback-to-last-working-query fires on 7% of episodes here against 41% there, and an untrained model's score is dominated by that fallback. The trained models barely use it (2-7%), which is why their numbers reproduce their training records to within a point or two. The honest reading is that the 0.290 figure is the one measured the same way as every other bar in the chart, and that an untrained baseline in this harness carries more measurement uncertainty than the trained ones do.

Does the published repo match what training measured?

Training could not measure this repo. Its vLLM engine was built from the base model with the checkpoint's weights synced in — a workaround forced by the fact that the raw training export is not loadable by vLLM at all. This repo is the rebuilt, directly-loadable one, so "same model" needed checking rather than assuming.

Both paths were run over the same 300 questions and compared paired (exact McNemar + bootstrap CI), which is the right test here because at n=300 the unpaired noise floor is several points:

mode	weight-sync path	this repo	paired delta	McNemar p
full	0.517	0.553	+0.037, 95% CI [-0.007, +0.080]	0.13
withheld	0.517	0.547	+0.030, 95% CI [-0.010, +0.070]	0.18

No significant difference in either mode, and the published path never scored lower. Read that as "consistent within about ±5 points", not as proof of equality — failure to reject is not equivalence, and the CI width is what the test can actually exclude.

Two caveats that apply to the absolute numbers regardless:

• The checkpoint was selected on BIRD. Steps 1250-1650 all sit in a 0.51-0.56 band, so treat ~0.55 full as the plateau rather than any single step as a clean result.
• This does not separate from its sibling. The two models differ by ~1 point, well inside the floor. Do not read the training-data comparison off these numbers; run a paired test if you need it.

Numbers are BIRD dev, produced by this project's own harness, and are not leaderboard submissions.

Serving with vLLM

The repo is a complete Qwen3_5ForConditionalGeneration checkpoint, so it loads directly — no conversion, no --hf-overrides:

vllm serve cwolff/qwen3.5-2b-grpo-bird --max-model-len 16384

from vllm import LLM, SamplingParams
llm = LLM(model="cwolff/qwen3.5-2b-grpo-bird", max_model_len=16384)
out = llm.chat(
    [{"role": "system", "content": SYSTEM_PROMPT},
     {"role": "user", "content": "How many singers are there?"}],
    SamplingParams(temperature=0.6, top_p=0.95, top_k=20, max_tokens=2048),
)

generation_config.json stops on both <|im_end|> (248046) and <|endoftext|> (248044), matching the stop set used during RL rollouts. The base model's own config lists only <|endoftext|>, which would run past the end of every assistant turn.

How to prompt it

This model expects an agent loop, and will underperform badly without one. It was trained under the exec_plan protocol: each assistant turn emits a single JSON object and nothing else — no prose, no code fences — and your harness executes it and feeds the result back as the next user turn.

{"tool": "list_tables"}
{"tool": "describe_table", "args": {"table_name": "singer"}}
{"tool": "run_query", "args": {"sql": "SELECT COUNT(*) FROM singer"}}
{"tool": "submit_sql", "args": {"sql": "SELECT COUNT(*) FROM singer"}}

Tools: list_tables, describe_table, foreign_keys, join_path, sample_rows, run_query, distinct_values, submit_sql. submit_sql ends the episode. Assistant turns open with a <think> block — keep thinking enabled; the chat template does this by default.

The exact system prompt matters. It is EXEC_PLAN_TOOL_INSTRUCTIONS in shared/tools/protocol.py of the SQaLe finetune repo — use that string rather than paraphrasing the tool list above, which is a summary.

Training

Base	Qwen/Qwen3.5-2B, stock (no SFT warm start)
Algorithm	GRPO, token-level loss, sum/1024 aggregation, no advantage normalisation
KL	k3 estimator toward a frozen reference, target 0.005
Clipping	sequence-level importance ratio from vLLM sampler logprobs, 0.2 / 0.28 (DAPO)
Batch	18 questions × 8 candidates = 144 episodes/step, 1 inner epoch
LR	1e-5, cosine, 5% warmup
Reward	dense; 3.0 semantic equivalence, F1 partial credit (scale 0.9), 0.1 join bonus, 0.1 consensus bonus
Environment	exec_plan, 6 tool turns, 12288-token episode budget, in-memory SQLite
Hardware	3× H100, 3-rank torchrun data parallel, 48 h wall clock
Steps	1350 of a planned 1800 (the run hit its wall-clock limit at ~1660)

Reward is computed by executing the candidate against a real SQLite database built from the row payloads and comparing to the gold result — not by string match against a reference query.

Limitations

• 2B parameters. It is a small model and will lose to a frontier model on hard multi-join questions.
• Trained and evaluated on SQLite only.
• Trained at a 6-turn budget with a 12288-token episode cap. Longer budgets are untested and truncation reached 9-22% at this one.
• Single-turn "schema in, SQL out" prompting is out of distribution — the withheld numbers above are what this model is actually for.
• Inherits the base model's vision tower, which is carried over untouched and was never trained or evaluated here. Treat it as a text model.

Provenance

Slurm job 26640987, output grpo1800_bird, checkpoint best_bird/ (step 1350), selected by the trainer on BIRD full official accuracy. Exported by merging the run's 320 trained language-model tensors with the base model's vision tower and MTP head, under the base config.json.