D1 Omni 600M Text2SQL Decisions

A fine-tune of LiquidAI/d1-omni-600M that serves as the decision model of the decisionmodel-text-to-sql app. It scores the options the app builds from the schema for an English question (measure, grouping, filters, order, plan); the app compiles the choices into PostgreSQL, so the model never writes SQL. Version 2.0 was trained on seven schemas and tested on held-out questions over them and on the unseen Chinook. Version 1.0 (Olist only) is under the v1.0 tag.

When the app returns a single answer it is right about 47% of the time on schemas outside Olist. Use it behind an interface where the user confirms or picks the interpretation.

Results

A question is strict when the app returned the reference result alone and correct or offered when it returned or listed it among offered interpretations. Accepted wrong means it returned a single different result. Single answers right is strict divided by strict plus accepted wrong. Rates are over answerable questions. Clef Flash (cloudflare/clef-flash) and D1 3B (liquid/d1-20260930) are hosted decision models called through OpenRouter's decisions API with the same app and options.

Chinook, a music store with 11 tables, was excluded from training (1,600 answerable questions, 400 to refuse):

Decision model	Strict	Correct or offered	Accepted wrong	Single answers right	Correctly refused
This model	29.6%	34.0%	31.8%	48%	89.2%
Clef Flash	28.1%	38.1%	14.9%	65%	95.0%
D1 3B	22.2%	33.6%	9.1%	71%	99.0%

Over Chinook and six training schemas (Pagila, Northwind, AdventureWorks, nycflights13, Synthea and HR; 11,200 answerable, 2,800 to refuse) each answerable question has one of five outcomes:

Decision model	Strict	Offered, list holds the answer	Offered, list misses it	Accepted wrong	Refused	Single answers right	Correctly refused
This model	27.0%	2.7%	4.6%	30.2%	35.5%	47%	91.2%
Clef Flash	23.0%	11.1%	10.3%	15.8%	39.8%	59%	95.2%
D1 3B	19.2%	7.8%	3.5%	9.2%	60.4%	68%	97.8%

Offered lists held 2.4 interpretations on average with this model, 3.6 with Clef Flash and 4.2 with D1 3B.

Accuracy at a matched error rate

This model's strict lead comes from answering more often. The curves hold the error rate fixed: accepted answers are ranked by the product of their decision confidences and the least confident are withdrawn first, counted as no answer. On the 6,675 answerable questions traced for all three models, at D1 3B's 9.4% accepted wrong this model keeps 16.6% strict (D1 3B 19.2%, Clef Flash 14.5%). At Clef Flash's 15.7% it keeps 21.8% against 22.7%. Ranking by the geometric mean or the minimum confidence gave lower curves in the same order. This model's accuracy falls with plan length, from 61% at 20 to 27 decisions to 5% beyond 46. Its less confident half of accepted answers averages 38 decisions against 34. Calibration fitted on generated data transfers poorly: expected calibration error is 0.001 on the 147,822 calibration decisions and 0.13 on the 3,886 accepted test answers (mean confidence 0.59, accuracy 0.47).

Chinook's 34.0% correct or offered sits inside the training schemas' range (18.9% to 35.3%), though as a widely published sample database it may appear in the base model's pretraining. On Olist, which has a hand-written bundle, this model reaches 48.7% correct or offered and 39.5% strict on 1,619 answerable questions (version 1.0: 37.6% and 24.0%; Clef Flash 52.1% and 23.7%; D1 3B 38.9% and 28.4%) and refuses 278 of 299 requests it should decline. SLM-SQL-1.5B scores 26.0% strict there. Olist baselines used app commit 24ab11a; everything else used 1ee9eb6.

Test questions and reference queries were written by a language model that never saw the training generator. A second model wrote an independent reference, a third settled the 15% to 29% of disagreements and every kept reference was rerun on the database. The sets are not public.

Models that write SQL

SLM-SQL fine-tunes Qwen2.5-Coder to write SQL directly. Both checkpoints answered every question with greedy decoding and their own prompt set to PostgreSQL, given each schema as CREATE TABLE statements with column notes and example values; sqlglot 26.17.1 translated their SQLite-style output. A request counts as refused when its SQL fails to run.

Model	Strict	Accepted wrong	Single answers right	Select	Detail list	Anomaly	Period change	Correctly refused
This model	27.0%	30.2%	47%	40.5%	2.9%	37.1%	27.3%	91.2%
SLM-SQL-1.5B	21.5%	43.9%	33%	51.2%	35.0%	0.0%	0.0%	58.2%
SLM-SQL-0.5B	16.2%	43.5%	27%	41.8%	23.1%	0.0%	0.0%	57.7%

SLM-SQL-1.5B leads on select and detail-list questions, which state what to compute; its 35.0% on detail lists suggests the decision models' weakness there lies in the options the app builds. Anomaly and period-change references follow the app's own definitions, which the questions do not spell out.

The two complement each other as answer and check. On the 8,360 questions with full traces, SLM-SQL received only the tables in this model's plan (prompts four times shorter) and drew four samples. This model's answer is returned when a sample gives the same result or its plan is an anomaly or period-change analysis; when this model gives no answer, a SQL answer is used only if all four samples agree.

System	Strict	Accepted wrong	Correctly refused
This model alone	27.3%	30.9%	92.1%
With SLM-SQL-0.5B	27.3%	14.7%	98.2%
With SLM-SQL-1.5B	28.8%	15.4%	97.1%

The check halves accepted wrong without losing strict answers; ranking this model's answers by confidence reaches 14.7% accepted wrong only at 21.0% strict. Using SQL whenever this model abstains instead raises strict to 32.7% and accepted wrong to 60.6%. Each sample spends about 500 tokens on reasoning, so the check is slow; phone timing is unmeasured.

Speed and precision

bf16/ holds the same weights in bfloat16. Variants were scored on the first 1,000 rows of the generated development set; a row is one app call of about 16 decisions and counts only when all are right, so it does not compare with the question-level results.

Weights	Hardware	File	Rows correct	Time per decision
float32	Colab GPU	2.35 GB	88.9%	23.4 ms
bfloat16	Colab GPU	1.17 GB	88.9%	6.0 ms
float32	Colab CPU, 12-thread Xeon at 2.2 GHz	2.35 GB	88.9%	311 ms
int8 encoder, dynamic	Colab CPU, 12-thread Xeon at 2.2 GHz	1.49 GB	47.1%	198 ms

bfloat16 chose the same option as float32 on 99.98% of decisions. Dynamic int8 quantization of the encoder lost 42 points; weight-only int8 is untested and no int8 file is released. At about 35 decisions per question the CPU rate means roughly 11 s of model time per question. On a Colab GPU behind a tunnel this model answered in a median 4.3 s end to end (Clef Flash 3.5 s, D1 3B 2.3 s). Laptops and phones are unmeasured. The decision server loads float32; for bfloat16 compute use AutoModel.from_pretrained(..., subfolder="bf16", dtype=torch.bfloat16, trust_remote_code=True).

Training

The 118,032 decision rows come from the seven training schemas, balanced across five question types, with every reference plan run on its database. The encoder and decision head (380.7M parameters) were trained for 8,000 steps with AdamW at a learning rate of 1e-5 in bf16 on one NVIDIA L4. The release is the step-7,000 checkpoint, best on the development set. Temperatures fitted on 147,822 held-out calibration decisions are stored in config.json.

Use

git clone https://github.com/Chulinuwu/decisionmodel-text-to-sql && cd decisionmodel-text-to-sql/local-model
pip install -r requirements.txt
hf download Chulinz/d1-omni-600m-text2sql-decisions --exclude "bf16/*" --local-dir models/text2sql-decisions
python -m decision_server --model-dir models/text2sql-decisions --device cuda --port 8790

Use --device mps on Apple silicon or --device cpu elsewhere. Then start the app from the repository root and open http://localhost:4317:

npm install && npm run setup
DECISIONS_URL=http://127.0.0.1:8790/decisions npm start

Limitations

The public app serves the Olist schema only; the multi-schema version used for testing is not yet published.

License

The weights modify LiquidAI/d1-omni-600M under the LFM Open License v1.0 (LICENSE, changes in NOTICE), which restricts commercial use above its revenue threshold. The training rows derive from Olist (CC BY-NC-SA 4.0), Pagila (PostgreSQL License), Northwind (Ms-PL), AdventureWorks (MIT), nycflights13 (CC0), Synthea (public domain) and the Oracle HR sample (MIT). Chinook (MIT) was used for testing only. Whether the non-commercial and share-alike terms of the Olist license extend to weights trained on derived rows is unsettled; check before commercial use.