KBBridge/KBBridge-v3-FP8

KBBridge-v3 (FP8)

A fine-tune of Qwen/Qwen3.8-27B specialised in GeneXus programming, in the native .gxSource export format.

Frontier models do not know this format. Without the GeneXus documentation injected into the prompt they produce syntactically invalid output almost every time (parse rate 0.5–3.1%). KBBridge writes it natively, runs on your own hardware, and never sends your Knowledge Base code to an external API.

---

⚠️ Read this before your first prompt

Two settings, or the model will look broken. Both are measured, not stylistic.

1. Turn reasoning OFF

# vLLM: pass it per request
#   "chat_template_kwargs": {"enable_thinking": false}

The Qwen chat template enables a <think> block by default. If that block does not close within your token budget, the answer comes back empty or half-finished — the client sees "the model did not respond". Measured on this GGUF: with reasoning on, 300 tokens were not enough to even begin the object; with it off, the same prompt returned a complete, valid Procedure.

Our own gateway sets this by default for exactly this reason.

2. Ask for the format explicitly

Write "in .gxSource format" in your prompt.

Measured on v3: the bare request "a Procedure that adds two numbers" returns generic SQL. Naming the format returns the GeneXus object, consistently. If you use a harness with its own system prompt, put the instruction there once.

3. Give it enough room

max_tokens ≥ 4096. A .gxSource object consumes roughly 340 tokens per KB of source, and most tools default to 512–1024, which truncates the object mid-body.

---

Results

580 held-out items (191 codegen + 329 MCQ + 60 data-model) that no model saw during training. Syntax validated with the official GeneXus ANTLR parser. Same protocol for every model: temperature 0.1, reasoning off, concurrency 8.

v3 vs v2 — an honest comparison

v3 is not a clean win over v2. It gains domain knowledge and loses syntax accuracy:

Metric	v2	v3
parseRate (valid syntax)	89.0	84.8	−4.2
parmMatch (exact signature)	78.6	78.6	=
MCQ (GeneXus knowledge)	76.0	79.0	+3.0
methodValidity	90.0	91.1	+1.1

What these numbers do NOT establish. v3 changed three things at once — the base model (Qwen3.6 → 3.8), the corpus (4× larger, per-KB cap removed) and the teacher (v1 → v2). The parseRate drop cannot be attributed to any one of them without a control arm that was never run. Anyone reading this table as "the bigger corpus hurt syntax" is over-reading it.

Choose v3 if domain knowledge matters more to you; v2 still leads on raw syntax validity.

Generalisation to unseen Knowledge Bases

Three entire KBs were held out — different domains, never in the pipeline:

	held-out from training KBs	3 completely new KBs
v2	89.0	89.9
v3	84.8	87.4

v3's relative gap to unseen KBs is larger than v2's (+2.6 vs +0.9), i.e. it generalises better in relative terms, even though two KBs make up 54.7% of its corpus.

Fairness note on the frontier comparison

In our benchmark the frontier models were run with ~21,600 tokens of GeneXus documentation injected into every request; KBBridge was run without any. That is not a handicap we imposed — injecting the same documentation into KBBridge makes it worse (76.4 → 73.3 parseRate), because the fine-tune already internalised that knowledge and the extra context gets in the way. Still, the setups differ, and you should know that when reading any head-to-head number.

About quantisation quality

The 4-bit GGUF build was measured against the bf16 master on the same 580 items, and excluding items where either run hit the token ceiling the two are indistinguishable (parseRate 93.0 vs 93.6 over 171 items). FP8 is a lighter quantisation than that, so the same conclusion applies with room to spare — though we did not benchmark this FP8 build separately.

---

Files

FP8 dynamic quantisation, 29 GB, single shard. Built for vLLM on Hopper/Blackwell-class GPUs. This is the build we run in production.

vllm serve KBBridge/KBBridge-v3-FP8 --served-model-name kbbridge-v3 \
  --max-model-len 262144 --kv-cache-dtype fp8 \
  --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder

Speculative decoding — enabled, and it needs one specific setting

The multi-token-prediction head is in this checkpoint (2 shards; the mtp.* tensors are kept in bf16 and listed in the quantisation ignore list). Turn it on with:

vllm serve KBBridge/KBBridge-v3-FP8 --served-model-name kbbridge-v3 \
  --max-model-len 262144 --kv-cache-dtype fp8 \
  --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --speculative-config '{"method":"mtp","num_speculative_tokens":1}'

Measured on one RTX PRO 6000, 301-token generations, first run discarded:

	median	vs baseline
no speculative decoding	47.5 tok/s	—
num_speculative_tokens: 1	53.7 tok/s	+13%
num_speculative_tokens: 2	45.4 tok/s	−4% — worse than off

Use 1, not 2. This model declares mtp_num_hidden_layers: 1, so with > 1 vLLM runs several forward passes over the same MTP layer; it warns about this in the log ("may result in lower acceptance rate") and we measured it landing below the no-speculation baseline. At 1, vLLM reports 97.4% draft acceptance.

Output is unchanged either way — the main model verifies every drafted token, so a draft head can only affect speed.

ninja must be on PATH. vLLM JIT-compiles a kernel for the speculative path on the first inference. Without it, the model loads completely — roughly nine minutes — and then the engine dies with FileNotFoundError: 'ninja'. pip install ninja in the vLLM environment.

Note for anyone re-quantising this model themselves

Re-quantising from bf16 drops the MTP head, silently. llmcompressor loads through transformers' Qwen3_5ForConditionalGeneration, which has no module for it, so the 15 mtp.* tensors are discarded on load with no warning — we re-ran the quantisation from a master that contains them and the output was byte-identical to one that never had them. They have to be spliced into the checkpoint afterwards and added to the quantisation ignore list.

Intended use

Assisting GeneXus developers: generating objects (Procedures, Transactions, Data Providers, SDTs, WebPanels), explaining existing code, completion, and documentation questions.

Out of scope: not a general-purpose model, not a replacement for validating in the GeneXus IDE, and it does not know any particular Knowledge Base (see Limitations).

---

Limitations

• It does not know your KB. It learned the style and syntax of the format, not the contents of any specific base. Ask it about a transaction you did not paste in, and it will invent plausible attribute names and present them as fact. Always give it the context and validate the output in the IDE.
• Runaway generation on very large objects. For objects over ~10 KB the model can fall into degenerate repetition — the same line hundreds of times without closing the object. Measured on v2 at ~1.6% of benchmark items; not re-measured on v3. Raising max_tokens does not fix it. Generate large objects section by section.
• Spanish bias in explanations, reflecting the corpus.
• Specialised: worse than the base model at general tasks.
• The limitations above other than the first were measured on v2 and are carried over as working assumptions, not verified properties of v3.

If you also use a hosted KBBridge endpoint

The raw GGUF and a gateway-fronted deployment do not behave the same by default. Our gateway applies four corrections the plain model does not have: a max_tokens floor, reasoning disabled, a reasoning_content fallback when content comes back empty, and repetition_penalty 1.05 to suppress runaway. If you compare "what I tried on your server" against "what I downloaded", the difference is those four settings, not the weights.

---

Training

Method	QLoRA 4-bit (bitsandbytes) + Liger kernel
LoRA	r=64, α=128, dropout=0.05, all projections
Context	12,288 tokens
Effective batch	16 (1 × 16 grad accum)
LR	1.0e-4, cosine, 3% warmup
Epochs	2 complete (14,108 steps)
Hardware	1× RTX PRO 6000 Blackwell 96 GB
Duration	7 days 4:41
Framework	LLaMA-Factory, transformers 5.6.0

train_loss 0.2618 (v2: 0.3344) · eval_loss 0.3723 (v2: 0.4675), minimum at the last step — no overfitting across 71 evaluations, which suggests there was room for more epochs.

Note that these losses are much better than v2's and yet parseRate went down: eval_loss measures fit to the corpus, not GeneXus quality.

Data

80,344 examples derived from GeneXus objects across 25 real Knowledge Bases (GX16/17/17U8/18/ Evo1, multi-domain) — 129% more than v2, with the per-KB cap removed. Sanitised, deduplicated and split by deterministic hash. The datasets are not published: they contain customer proprietary code.

---

Training-data privacy

The model was trained on real customer Knowledge Bases, so we audited whether it can leak them. This is the strongest result of the project.

Canaries: no memorisation threshold found

12 synthetic objects containing unguessable 16-character secrets were inserted at four frequencies, and verified to have reached train.jsonl at exactly those counts:

repetitions	canaries	recovered by name	recovered with literal prefix
1	3	0/3	0/3
10	3	0/3	0/3
100	3	0/3	0/3
1000	3	0/3	0/3

Not even at a thousand identical repetitions. A control rules out a broken probe: asked for the canary, the model returns a structurally valid but empty object — no token, no secret. And it does generate real bodies when the request has content, so the empty skeleton is not an inability to generate.

Membership inference: marginal signal

mean loss, seen examples	3.4130
mean loss, unseen	3.7711
mean length	3,133 vs 3,117 chars — comparable, so the AUC is meaningful
AUC	0.5539

0.554 against 0.50 for indistinguishable. There is a statistical trace of having seen the data, but the distributions overlap almost entirely.

Conclusion: customer code is not recoverable from the weights.

Caveat, stated plainly: absence of evidence is not proof of absence. These audits cover the attacks we ran, not every attack that exists.

---

Reproducibility

Full external reproduction is not possible, and it is worth saying so directly:

1. The 25 Knowledge Bases are customer code and are not distributed.
2. The parseRate scorer uses the KBEditor's ANTLR parser — proprietary, not distributable.
3. The teacher that generated v3's data is KBBridge-v2, which is not published.

What a third party can verify: the raw benchmark outputs (one model response per item) and the scoring over them.

---

Citation

@misc{kbbridge-v3,
  title  = {KBBridge-v3: a GeneXus code assistant fine-tuned from Qwen3.8-27B},
  author = {{KBBridge}},
  year   = {2026},
  url    = {https://huggingface.co/KBBridge/KBBridge-v3-FP8}
}

License

Apache 2.0, inherited from the base model Qwen/Qwen3.8-27B. This is a modified derivative work; see NOTICE.