JEB-35B-A3B

Research model, no warranty. JEB-35B-A3B is a decision model: it answers typed questions about a state (yes/no, a choice among options, a score on a scale) with calibrated probabilities from a single forward pass, without generating text. All questions of a request share the state's prefix and are scored in parallel. It is served through JEB, an open server with a System One-compatible API.

Fine-tuned from Qwen/Qwen3.6-35B-A3B by szybkie.ai with a decision objective: a restricted softmax over the answer tokens is trained toward soft targets distilled from a 176B-parameter teacher, so the model learns to get the probabilities right, not only the argmax. This checkpoint is a mixture-of-experts model (35B parameters, about 3B active per token); it serves in FP8 on a single 80 GB-class GPU or a DGX Spark.

Numbers (round 3, 2026-09-20)

Accuracy on held-out public sets (validation/test splits, never trained on), raw probabilities (no post-hoc calibration); ECE in parentheses where available. Full tables, reliability diagrams and the Doom/Atari transfer results are in the technical report.

set	n	base Qwen3.6-35B-A3B	JEB-35B-A3B (round 3)
AG News	300	0.877 (0.091)	0.913 (0.019)
SST-2	300	0.957 (0.024)	0.960 (0.009)
TREC	300	0.937 (0.034)	0.973 (0.026)
BoolQ	300	0.880 (0.047)	0.920 (0.030)
CLINC150 (20 options)	400	0.968 (0.009)	0.980 (0.029)
QNLI	300	0.933 (0.036)	0.920 (0.037)
RTE	277	0.874 (0.040)	0.903 (0.022)
IMDB	300	0.960 (0.023)	0.957 (0.022)
DBpedia	300	0.983 (0.011)	0.973 (0.030)
ARC-Easy	300	0.990 (0.012)	0.987 (0.017)
MMLU (500)	500	0.818 (0.059)	0.838 (0.034)
STS-B	300	0.470 (0.168)	0.590 (0.075)
blackjack (basic strategy)	300	0.713 (0.171)	0.713 (0.096)
tic-tac-toe (minimax)	300	0.367 (0.188)	0.383 (0.054)
document retrieval (which passage / answerable)	300	0.930 (0.053)	0.953 (0.014)
claim supported by passage	172	0.983 (0.032)	0.988 (0.017)
MedQA (USMLE) test	300	0.880 (0.022)	0.897 (0.071)
differential over candidate conditions (+ 'none listed')	300	0.520 (0.304)	0.637 (0.058)
all 5,549 items		0.834 (0.062)	0.859 (0.009)

Run it

pip install git+https://github.com/szybkie-ai/jeb
vllm serve szybkie-ai/jeb-35b-a3b --served-model-name jeb-base --quantization fp8 --max-logprobs 32 --port 8021
jeb serve --engine vllm-http --engine-url http://127.0.0.1:8021 --engine-model jeb-base --tokenizer szybkie-ai/jeb-35b-a3b
curl localhost:8020/v1/systemone -H 'Content-Type: application/json' -d '{"state": "I was charged twice for order A-104.",
  "model": "jeb-latest", "questions": {"refund": {"type": "noul", "instructions": "Does the customer ask for a refund?"}}}'

Or docker compose up in the JEB repository (vLLM + server).

Intended use and limitations

JEB is a research model, published so that the method, the weights and the numbers can be examined and built upon. It is not a product and comes with no warranty or support. Intended use: classification, routing, extraction of typed judgements, and any decision inside software where a probability is more useful than a sentence; the output is a distribution over the options you give it, and it does not generate text.

Probabilities are calibrated on the evaluation mix above; on other distributions, measure calibration on your own data and set your own thresholds before relying on it. The medical rows in the training and evaluation mix exist to study calibration on hard multi-option decisions; the model must not be used for medical, legal, financial or safety decisions without independent validation and human review. Knowledge is that of the base model: the fine-tune changes how the model decides and how honest its probabilities are, not what it knows.

Training

Base Qwen/Qwen3.6-35B-A3B; LoRA on the attention and gated-delta projections and, per expert, on the expert weights of every MoE layer; one epoch of ~42.5k rows; teacher-labelled soft targets ("mix": half gold one-hot, half teacher distribution); cosine schedule. Data: public classification and entailment sets, knowledge MCQ (ARC, OpenBookQA, SciQ, CommonsenseQA, MMLU auxiliary train); document question answering over Polish and English passages (multiple choice with gold, claim verification, passage retrieval); medical exam-style questions (MedQA) and differential-diagnosis cases built from published case reports, used for research on calibration only.

License: Apache-2.0 (weights), MIT (code). Made by szybkie.ai.