JEV-Qwen3.5-0.8B-Base

Jev-style text choice inference using Qwen3.5. This is not official TypeSafe Jev. "Base" means our baseline choice implementation before task-specific post-training, not a claim that the upstream checkpoint is a pretraining-only model.

Contents and provenance

Weights and tokenizer are unchanged from Qwen/Qwen3.5-0.8B. Original BF16 checkpoint, cast to FP16 at runtime; recurrent SSM state remains FP32. The custom SGLang runtime extracts 86 original vocabulary rows into a compact output head, skips vision weights and removes the full LM output head at load time. The checkpoint on disk retains upstream tensors for compatibility and provenance. For tied embeddings, the compact head is extracted from the embedding matrix. No ad-blocking, Snake RL, RLCD, or probability-calibration training is included.

Interface

Input: a question and 2–86 option strings. One text forward pass, no text decoding. Labels: 0–9, a–z, A–Z, αβγδεζηθικλμνξοπρστυφχψω. Input limit: 8192 tokens including the template and options. Output: selected index (1-based), label, option and all option probabilities. Softmax probabilities are conditional on the supplied options and are not calibrated correctness estimates. Use the included runtime; the standard Transformers generation pipeline does not implement this API.

Run

Requires Linux, a compatible NVIDIA GPU and SGLang with Qwen3.5 support. The 9B NVFP4 path requires native FP4 support (verified on RTX PRO 6000 Blackwell). Verified source runtime: SGLang commit e161bd1265a0082478b7f1c09f224a52d315dc71, PyTorch 2.13.0/CUDA 13.0, Transformers 5.12.1, FlashInfer 0.6.17, compressed-tensors 0.18.0; FastAPI, Uvicorn and Pydantic 2 are also required. CUDA kernels may compile on first launch. These scripts use this SGLang revision's embedding and prefill-graph APIs; compatibility with other revisions is not guaranteed.

From the downloaded repository directory, in that environment:

MAX_JOBS=3 python serve.py --port 8231

curl http://127.0.0.1:8231/v1/choice -H 'Content-Type: application/json' \
  -d '{"question":"请选择猫","options":["狗","猫"]}'

For one request followed by shutdown: python serve.py --request request.json. GPU memory fraction defaults to 0.2; override with JEV_GPU_MEMORY_FRACTION for your hardware. The source deployment has been exercised locally. Functional examples and option-boundary checks are not a broad accuracy benchmark. No ad-detection effectiveness is claimed.

Attribution

Qwen3.5 by the Qwen team; 9B NVFP4 quantization by IG1 where applicable. The included baseline inference adaptation is released under Apache-2.0. See LICENSE and upstream documentation. No upstream model training is attributed to this release.

JevBench public self-evaluation (2026-09-22)

147/231 correct (63.64%), public subset only; not an official full-suite JevBench score or ranking. No task-specific post-training, prompt tuning or temperature fitting.

Tier	Correct	Accuracy
Easy	45/48	93.75%
Standard	50/72	69.44%
Hard	52/111	46.85%

All 231 requests/distributions were valid. Overall Brier 0.559500, 10-bin ECE 0.167781; these are empirical public-set metrics, not a calibration guarantee. Local HTTP p50 15.13 ms, p95 27.09 ms on RTX PRO 6000 Blackwell, with no observed concurrent training. Serial requests, 0.2 s pacing (excluded from latency); first request included.

Report · Reproduction · Per-item results · Aggregates