JEB-4B

Research model, no warranty. JEB-4B is a decision model: it answers typed questions about a state (yes/no, a choice among options, a score on a scale) with calibrated probabilities from a single forward pass, without generating text. All questions of a request share the state's prefix and are scored in parallel. It is served through JEB, an open server with a System One-compatible API.

Fine-tuned from Qwen/Qwen3.5-4B by szybkie.ai with a decision objective: a restricted softmax over the answer tokens is trained toward soft targets distilled from a 176B-parameter teacher, so the model learns to get the probabilities right, not only the argmax. This is the small member of the family: a dense 4B model that serves in FP8 on one consumer GPU (about 5 GB of weights). Its bigger sibling, JEB-35B-A3B, is trained on the same data and is stronger on knowledge-bound questions; the table shows both.

Numbers (round 3, 2026-09-20)

Accuracy on held-out public sets (validation/test splits, never trained on), raw probabilities (no post-hoc calibration); ECE in parentheses where available. Full tables, reliability diagrams and the Doom/Atari transfer results are in the technical report.

set	n	base Qwen3.5-4B	JEB-4B	JEB-35B-A3B	Jev (hosted)
AG News	300	0.857 (0.105)	0.917 (0.043)	0.913 (0.019)	0.863 (0.095)
SST-2	300	0.933 (0.050)	0.953 (0.021)	0.960 (0.009)	0.957 (0.102)
TREC	300	0.773 (0.100)	0.963 (0.021)	0.973 (0.026)	0.933 (0.025)
BoolQ	300	0.847 (0.044)	0.887 (0.031)	0.920 (0.030)	0.917 (0.022)
CLINC150 (20 options)	400	0.960 (0.017)	0.960 (0.038)	0.980 (0.029)	0.973 (0.012)
QNLI	300	0.843 (0.053)	0.883 (0.015)	0.920 (0.037)	0.930 (0.044)
RTE	277	0.841 (0.064)	0.892 (0.023)	0.903 (0.022)	0.913 (0.042)
IMDB	300	0.947 (0.018)	0.950 (0.033)	0.957 (0.022)	0.970 (0.049)
DBpedia	300	0.987 (0.017)	0.980 (0.036)	0.973 (0.030)	0.983 (0.015)
ARC-Easy	300	0.980 (0.030)	0.967 (0.023)	0.987 (0.017)	0.993 (0.007)
MMLU	500	0.730 (0.052)	0.706 (0.049)	0.838 (0.034)	0.920 (0.026)
blackjack (basic strategy)	300	0.577 (0.050)	0.543 (0.055)	0.713 (0.096)	0.837 (0.125)
tic-tac-toe (minimax)	300	0.390 (0.067)	0.393 (0.071)	0.383 (0.054)	0.427 (0.065)
document retrieval	300	0.897 (0.023)	0.947 (0.018)	0.953 (0.014)	0.910 (0.046)
claim vs passage	172	0.866 (0.065)	0.988 (0.015)	0.988 (0.017)	0.977 (0.028)
document MCQ (Polish)	129	1.000 (0.018)	0.992 (0.011)	1.000 (0.007)	1.000 (0.003)
document MCQ (Polish, medical)	33	0.970 (0.028)	0.970 (0.024)	1.000 (0.019)	1.000 (0.001)
document MCQ (English, medical)	8	1.000 (0.033)	1.000 (0.012)	1.000 (0.006)	1.000 (0.000)
MedQA (USMLE)	300	0.683 (0.067)	0.643 (0.092)	0.897 (0.071)	0.843 (0.050)
differential (6-8 candidates + none)	300	0.480 (0.257)	0.460 (0.093)	0.637 (0.058)	0.537 (0.213)
all 5,419 items		0.804 (0.038)	0.825 (0.007)	0.878 (0.008)	0.880 (0.014)

Run it

pip install git+https://github.com/szybkie-ai/jeb
vllm serve szybkie-ai/jeb-4b --served-model-name jeb-base --quantization fp8 --max-logprobs 32 --port 8021
jeb serve --engine vllm-http --engine-url http://127.0.0.1:8021 --engine-model jeb-base --tokenizer szybkie-ai/jeb-4b
curl localhost:8020/v1/systemone -H 'Content-Type: application/json' -d '{"state": "I was charged twice for order A-104.",
  "model": "jeb-latest", "questions": {"refund": {"type": "noul", "instructions": "Does the customer ask for a refund?"}}}'

Or docker compose up in the JEB repository (vLLM + server).

Intended use and limitations

JEB is a research model, published so that the method, the weights and the numbers can be examined and built upon. It is not a product and comes with no warranty or support. Intended use: classification, routing, extraction of typed judgements, and any decision inside software where a probability is more useful than a sentence; the output is a distribution over the options you give it, and it does not generate text.

Probabilities are calibrated on the evaluation mix above; on other distributions, measure calibration on your own data and set your own thresholds before relying on it. The medical rows in the training and evaluation mix exist to study calibration on hard multi-option decisions; the model must not be used for medical, legal, financial or safety decisions without independent validation and human review. Knowledge is that of the base model: the fine-tune changes how the model decides and how honest its probabilities are, not what it knows.

Training

Base Qwen/Qwen3.5-4B; LoRA on all linear projections of all 32 layers; one epoch of ~42.5k rows; teacher-labelled soft targets ("mix": half gold one-hot, half teacher distribution); cosine schedule. Data: public classification and entailment sets, knowledge MCQ (ARC, OpenBookQA, SciQ, CommonsenseQA, MMLU auxiliary train); document question answering over Polish and English passages (multiple choice with gold, claim verification, passage retrieval); medical exam-style questions (MedQA) and differential-diagnosis cases built from published case reports, used for research on calibration only.

License: Apache-2.0 (weights), MIT (code). Made by szybkie.ai.