JEV-Qwen3.5-9B-Base-NVFP4

Jev-style text choice inference using Qwen3.5. This is not official TypeSafe Jev. "Base" means our baseline choice implementation before task-specific post-training, not a claim that the upstream checkpoint is a pretraining-only model.

Contents and provenance

Weights and tokenizer are unchanged from ig1/Qwen3.5-9B-NVFP4. Upstream compressed-tensors NVFP4 checkpoint; compact output head and sensitive layers remain BF16. The custom SGLang runtime extracts 86 original vocabulary rows into a compact output head, skips vision weights and removes the full LM output head at load time. The checkpoint on disk retains upstream tensors for compatibility and provenance. For tied embeddings, the compact head is extracted from the embedding matrix. No ad-blocking, Snake RL, RLCD, or probability-calibration training is included.

Interface

Input: a question and 2–86 option strings. One text forward pass, no text decoding. Labels: 0–9, a–z, A–Z, αβγδεζηθικλμνξοπρστυφχψω. Input limit: 16384 tokens including the template and options. Output: selected index (1-based), label, option and all option probabilities. Softmax probabilities are conditional on the supplied options and are not calibrated correctness estimates. Use the included runtime; the standard Transformers generation pipeline does not implement this API.

Run

Requires Linux, a compatible NVIDIA GPU and SGLang with Qwen3.5 support. The 9B NVFP4 path requires native FP4 support (verified on RTX PRO 6000 Blackwell). Verified source runtime: SGLang commit e161bd1265a0082478b7f1c09f224a52d315dc71, PyTorch 2.13.0/CUDA 13.0, Transformers 5.12.1, FlashInfer 0.6.17, compressed-tensors 0.18.0; FastAPI, Uvicorn and Pydantic 2 are also required. CUDA kernels may compile on first launch. These scripts use this SGLang revision's embedding and prefill-graph APIs; compatibility with other revisions is not guaranteed.

From the downloaded repository directory, in that environment:

MAX_JOBS=3 python serve.py --port 8231

curl http://127.0.0.1:8231/v1/choice -H 'Content-Type: application/json' \
  -d '{"question":"请选择猫","options":["狗","猫"]}'

For one request followed by shutdown: python serve.py --request request.json. GPU memory fraction defaults to 0.6; override with JEV_GPU_MEMORY_FRACTION for your hardware. The source deployment has been exercised locally. Functional examples and option-boundary checks are not a broad accuracy benchmark. No ad-detection effectiveness is claimed.

Attribution

Qwen3.5 by the Qwen team; 9B NVFP4 quantization by IG1 where applicable. The included baseline inference adaptation is released under Apache-2.0. See LICENSE and upstream documentation. No upstream model training is attributed to this release.

JevBench public self-evaluation (2026-09-22)

177/231 correct (76.62%), public items only; not an official full-suite JevBench score.

Tier	Correct	Accuracy
Easy	48/48	100.00%
Standard	68/72	94.44%
Hard	61/111	54.95%

All 231 requests and distributions were valid. Overall Brier 0.323008; 10-bin ECE 0.047818. These are observed public-set metrics, not a calibration guarantee. No task-specific training, prompt tuning or temperature fitting was performed.

Local HTTP p50 50.12 ms, p95 189.16 ms, measured while unrelated GPU training was running on the same RTX PRO 6000 Blackwell. One request at a time, 0.5 s pacing; the pacing/guard waits are excluded from latency. Do not compare this as idle-server or official leaderboard latency.

Report · Reproduction · Per-item results · Aggregates