jevhome/jevhome-B-int8

jevhome B int8

A typed-decision model (cross-encoder, ~150M parameters, int8) for jevhome: give it a state and yes/no (noul), pick-one (choice) or graded (score) questions, and it returns calibrated probabilities in one forward pass, on a plain CPU. Same API as Jev (POST /v1/systemone).

The order of the options of a choice question does not change the answer: all options get the same position ids, so the network reads them as a set, not a sequence, and a permutation gives the same probabilities up to floating-point rounding (int8 rounding can rarely change a close call).

This is the int8 version of jevhome/jevhome-B: dynamic int8 quantisation (ONNX Runtime quantize_dynamic) with 7-bit weights, the feed-forward output layers kept in fp32, and an int8 token-embedding table. The 7-bit weights keep it working on CPUs without AVX-512 VNNI (e.g. AMD EPYC Rome). It is faster and smaller, but it is a different model: it gives the fp32 answer on 97% of our evaluation items, and its accuracy below was measured on its own.

Run it

git clone https://github.com/Jibril-Frej/jev-at-home && cd jev-at-home && cargo build --release    # the jevhome binary (needs Rust)
hf download jevhome/jevhome-B-int8 --local-dir models/jevhome-B-int8
./target/release/jevhome serve models/jevhome-B-int8

Numbers

JevBench Easy	Standard	Hard	ECE (Easy+Standard, lower is better)	External mean (10 sets)	Latency p50	p90	Peak RAM
98	82	39	0.114	65.2	42 ms	120 ms	0.4 GB

Accuracy in %, measured with the jevhome binary. Latency: one decision at a time, 4 threads on 4 cores of an AMD EPYC 9654. The full tables (every set, Jev, Laya and the teacher), the training recipe and the data are in the jevhome README.

Files

• model.onnx: the network, probabilities come from calibrated logits.
• decision_config.json, tokenizer*.json, config.json: input format and tokenizer.
• calibration_u.json: one temperature per question type and number of options.

Licence

CC BY-NC 4.0 (non-commercial): several training sources are non-commercial or share-alike. Backbone: answerdotai/ModernBERT-base.