vonkaiser/Biomni-R0-32B-Preview-FP8-Dynamic

Biomni-R0-32B-Preview FP8 Dynamic

FP8_DYNAMIC compression of biomni/Biomni-R0-32B-Preview revision 71432eb3d5e583bee757e0f9437a17e711e8e3d1. This is not an official Biomni release. The original weights stay on that repo.

• Scheme: compressed-tensors FP8, per-channel weights, dynamic per-token activations
• lm_head left dense
• No calibration dataset
• Weights about 35GB

vLLM reads the scheme from config.json. Do not pass --quantization.

vllm serve vonkaiser/Biomni-R0-32B-Preview-FP8-Dynamic \
  --gpu-memory-utilization 0.90 \
  --kv-cache-dtype fp8 \
  --max-model-len 32768

The chat template asks for a JSON object inside <tool_call> tags. Use a tool parser that reads that format (qwen or hermes).

Base architecture is Qwen/Qwen3-32B. License MIT, same as the source model.

Comparison with the original

The original checkpoint and this FP8 checkpoint were scored on the same prompts. Both ran in vLLM at temperature 0 and seed 1234. Thinking was off, except one GSM8K slice. The original ran in BF16 with a context of 8192. FP8 used the compressed-tensors config, an FP8 KV cache, and a context of 32768. No Eval1 item hit a context limit on either side.

A gap of about one standard error is treated as the same result. The differences below are a few questions, and they do not all favor the same checkpoint.

GSM8K

Full set, 1319 questions, 5-shot, thinking off. One standard error is about 11 questions.

filter	original	FP8
strict-match	1179/1319 (0.8939)	1181/1319 (0.8954)
flexible-extract	1185/1319 (0.8984)	1189/1319 (0.9014)

Thinking on, first 200 questions. The comparison uses flexible-extract. One standard error on this slice is about 4 questions.

filter	original	FP8
flexible-extract	180/200 (0.900)	178/200 (0.890)

MMLU generative

5-shot, 5 questions from each subject, 285 questions. One standard error is about 6 questions.

	correct	rate
original	248/285	0.8702
FP8	250/285	0.8772

HumanEval

Chat pass@1 on 164 problems. Generated Python was executed. One standard error is about 4 problems.

	correct	rate
original	150/164	0.9146
FP8	148/164	0.9024

Biomni Eval1

All 433 rows were scored on both sides.

task	n	original	FP8
crispr_delivery	10	3	2
gwas_causal_gene_gwas_catalog	50	21	18
gwas_causal_gene_opentargets	50	38	38
gwas_causal_gene_pharmaprojects	50	28	32
gwas_variant_prioritization	43	2	3
lab_bench_dbqa	50	5	4
lab_bench_seqqa	50	7	8
patient_gene_detection	50	0	2
rare_disease_diagnosis	30	0	0
screen_gene_retrieval	50	19	19

Catalog is down 3 and pharma is up 4, each about one standard error on 50 questions. The other tasks move by one or two questions or stay tied. rare_disease_diagnosis is 0/30 on both sides.

Logprobs

160 prompts, compared on the shared top-1 prefix.

• Identical greedy completions: 110/160
• Aligned positions: 6812
• Mean absolute logprob difference: 0.008
• Largest absolute difference: 0.581
• Positions with a gap over 1.0 nat: 0
• First-token mismatches: 7/160

Tool calls

20 prompts: 16 that should call a tool, and 4 that should not. A call counts as correct when the tool name matches. Arguments are compared only when both sides called a tool.

Both checkpoints named the right tool on 16/16 and stayed quiet on 4/4. Arguments matched on 15 of those 16 calls. The one mismatch was "Calculate 15 percent of 80": the original passed 15% of 80 and FP8 passed 0.15 * 80.

Capability Deviation Score

+0.3 percentage points. FP8 scored slightly higher than the original.

The score is the gap between the two accuracy rates on the graded items counted once: GSM8K flexible-extract, MMLU, HumanEval, and Eval1 (2,201 items). The original was correct on 1,706 (77.5%). FP8 was correct on 1,713 (77.8%). A negative score would mean FP8 was less capable.

Each task gap in the tables above sits inside about one standard error. On the biomedical set the original scored 123 of 433 and FP8 scored 126 of 433. The largest moves there are the GWAS catalog task, down 3 of 50, and the pharma task, up 4 of 50.

The thinking-on GSM8K slice is left out of the 2,201 because those questions are already in the full GSM8K set. On that slice FP8 was 2 questions lower, 178 versus 180, which is inside the wider error on 200 questions.