You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Checkpoint released to reproduce the QATFactory paper (arXiv 2609.39223). Base model Qwen/Qwen3.5-9B (Apache-2.0).

Qwen3.5-9B NVFP4 QAD, Finding 4 Code-32K arm (QATFactory)

Qwen3.5-9B quantized to NVFP4 and recovered with quantization-aware distillation (QAD) on the code split of the Finding 4 corpus, truncated to 32K tokens. One of the four token-matched arms of the QATFactory paper's sequence-length and domain study (Table 7: 32K sequences, code domain), reproduced with the released code and regenerated corpus. Weights were fake-quantized during training (W4A16); the table reports the checkpoint served both W4A16 and W4A4, as in the paper.

Serve with vLLM (compressed-tensors nvfp4-pack-quantized; native NVFP4 needs Blackwell GPUs):

vllm serve qatfactory/Qwen3.5-9B-NVFP4-QAD-Code-32K --max-model-len 65536

Training

Recipe: configs/llm/qwen3_5_9b_nvfp4_qad_generalization_code_32k.yaml at QATFactory commit 39d259294f922e661473b621b2308803a862443b (in the history of main), run on 4 GPUs with --train.gradient_accumulation_steps 4 (same global batch 16 as the recipe's 8 GPUs x 2) and --model.attn_implementation sdpa, --train.max_steps 1819: the step count that matches the 4K arm's supervised tokens on the published corpus (scripts/generalization/); the paper's 1,817 were computed on its own copy of the corpus.
Data: qatfactory/openperfectblend-generalization-qwen3.5-9b-think-32k at revision b78b2ad8c7cc65cbcccb7274a94314fde318bdad (file sha256 48d1beaa6b06474360198d195889d0ceeefc477420b581b98965acd7b02848f4).
Hardware and software: 1 node of 4x NVIDIA GB300, image vllm/vllm-openai:v0.30.0-aarch64 (sha256:4864d46625cbc3307623e29ac742030655e27249feba7b97ec925ce4cc4dfb56): torch 2.13.0+cu130, transformers 5.17.0, accelerate 1.15.0, plus flash-linear-attention 0.6.0 and datasets 5.0.0.
Run: 1,819 optimizer steps at global batch 16 on the 45,000-example code split built by scripts/generalization/ (file sha256 9032eaff5c8e2386d620370b17e69cb5685506552ed34ff2c1d74aaa79550aa5), about 307.6M supervised tokens (matched between the 4K and 32K arms of a domain), lr 1e-6 cosine, forward KL to the frozen BF16 teacher, weights fake-quantized to NVFP4; trained over several 4-hour jobs, each resuming from the last checkpoint. Final held-out forward KL 0.0089 and teacher-student top-1 agreement 0.969.

Evaluation

Served with vLLM 0.30.0 on GB300, data-parallel 4, max-model-len 65536. W4A4: this checkpoint as published (native NVFP4 W4A4). W4A16: the same packed weights with input_activations removed from the quantization config and the input_global_scale tensors dropped, which vLLM serves through its NVFP4 Marlin W4A16 path. Evaluated with the released harness (scripts/eval/quality/, evalscope 1.9.1, temperature 1.0, top-p 0.95, top-k 20, 32K new tokens): AIME25 (5 samples per problem), BigCodeBench (300 of 1,140) and LiveCodeBench v6 (300). The BF16 column was measured with the same harness.

Benchmark	BF16	This QAD, W4A16	Paper, W4A16	This QAD, W4A4	Paper, W4A4
AIME25	50.0	52.0	52.00	48.0	46.67
BigCodeBench	37.0	38.0	34.00	34.0	24.33
LiveCodeBench v6	87.3	82.7	82.67	81.3	78.33
Average	58.1	57.6	56.2	54.4	49.8

LiveCodeBench is graded with evalscope's local executor (no Docker on the evaluation cluster). BigCodeBench check programs are built by the harness unchanged and executed in an aarch64 rebuild of the official bigcodebench-evaluate image (no 4 GB memory cap; tensorflow does not import, which affects 1 of the 300 sampled tasks); its absolute scores run about 5 points above the paper's, so compare within the column. Single-sample benchmarks of 300 questions carry 2-3 points of sampling noise. The BF16 column is the Qwen3.5-9B BF16 run of this series.

Provenance

Files under provenance/: trainer_state.json (loss, held-out KL and top-1 history), versions.txt (training environment).

Downloads last month: -

Safetensors

Model size

9B params

Tensor type

BF16

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for qatfactory/Qwen3.5-9B-NVFP4-QAD-Code-32K

Base model

Qwen/Qwen3.5-9B-Base

Finetuned

Qwen/Qwen3.5-9B

Quantized

(677)

this model