Qwen3.8-27B-bf16

The unquantized arm of a three-arm panel: Qwen/Qwen3.8-27B's text tower, QLoRA fine-tuned on text-to-SQL and merged back to bf16. It is published because the two DynQuant arms' headline numbers are differences against this model, measured in the same run on the same items -- and a difference whose reference nobody can download is not a measurement anybody can check.

You may want a smaller arm

This repo is 50.10 GiB. All three arms were scored in the same run, on the same 400 held-out items, at the same decode settings, and paired item by item (McNemar):

model	on disk	accuracy	vs bf16	95% CI	separated?
this repo (bf16)	50.10 GiB	85.50%	--	--	--
VikramPal/Qwen3.8-27B-DynQuant-4bit	12.54 GiB	84.25%	-1.25 pts	[-2.87, +0.37]	no (p = 0.2266)
VikramPal/Qwen3.8-27B-DynQuant-3bit	9.41 GiB	79.50%	-6.00 pts	[-8.71, -3.29]	yes (p = 1.93e-05)

"Separated: no" means the paired test could not tell that arm apart from this one at this sample size. That is not a claim that the two are identical -- read the confidence interval, which is the range of differences the data is consistent with.

What this is

• Base: Qwen/Qwen3.8-27B, text tower only. The vision tower is not fine-tuned and not evaluated here.
• Fine-tune: qlora, LoRA rank 32, 1.0 epoch at lr 0.0001, effective batch 16 over 625 steps, train loss 0.0963.
• Data: 9,999 conversations from spider, gretel, wikisql, create-context, 350,799 supervised tokens (loss is taken on the answer only).
• Merged: the adapter is folded into the base weights, so this is a plain bf16 checkpoint with no PEFT dependency at load time.
• Contamination: the training split was matched against every evaluation item and 601 examples were dropped for colliding with one. No source overlaps an evaluation task after that.

How it was scored

85.50% (342/400) on held-out text-to-SQL drawn from spider, gretel, wikisql, 2-shot, execution-free logic match. Decode was greedy with a budget of 1024 new tokens, and 0 generations reached it without finishing. 0 predictions were unparseable.

generation_config.json in this repo pins greedy decode, because that is how the number above was measured. If you sample, you are not running the configuration that produced it.

Use

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("VikramPal/Qwen3.8-27B-bf16")
model = AutoModelForCausalLM.from_pretrained(
    "VikramPal/Qwen3.8-27B-bf16", dtype=torch.bfloat16, device_map="auto"
)

schema = "CREATE TABLE singer (singer_id INT, name TEXT, age INT)"
prompt = f"Given the schema: {schema}\nQuestion: How many singers are there?\nSQL:"
enc = tok.apply_chat_template(
    [{"role": "user", "content": prompt}],
    tokenize=True, add_generation_prompt=True,
    return_tensors="pt", return_dict=True,
).to(model.device)
out = model.generate(**enc, max_new_tokens=256, do_sample=False)
print(tok.decode(out[0][enc["input_ids"].shape[1]:], skip_special_tokens=True))

The base is a reasoning model and its answers open with a </think> marker before the SQL. The scorer cuts at the first SELECT; if you consume the output programmatically, do the same rather than assuming the first line is the query.

It also serves as published: vllm serve VikramPal/Qwen3.8-27B-bf16, measured on vLLM 0.27.1. The quantized arms above do not, for a reason their own cards give.

Citation

@software{dynquant,
  author  = {Pal, Vikram},
  title   = {{DynQuant}: dynamic-signal quantization for large language models},
  url     = {https://github.com/kambojvikram/dynquant},
  year    = {2026}
}

@misc{qwen3_8_27b_text2sql_bf16,
  author       = {Pal, Vikram},
  title        = {Qwen3.8-27B-bf16},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/VikramPal/Qwen3.8-27B-bf16}}
}

Limits

• Scored on 400 items. Differences of well under a point are not resolvable at that size, which is why the table above reports intervals and a paired test rather than two accuracies side by side.
• Logic match is not execution accuracy: it compares query structure, not results against a live database.
• English prompts only; the fine-tune added no other language.
• Text only. The base model's vision tower is untouched by this fine-tune and its behaviour here is unmeasured.