Qwick-3.5-9B

日本語

Qwick-3.5-9B is a Qwen3.5-9B derivative selected for shorter reasoning traces on MMLU-Pro and GPQA-Diamond. It is a standalone BF16 Transformers checkpoint and loads directly from horiuchinobuyuki/Qwick-3.5-9B.

Developed by Nobuyuki Horiuchi.

Why Qwick

Qwen3.5-9B is a strong, broadly capable model that we use heavily, but in everyday use its reasoning tends to run long: it often spends many thinking tokens even on questions that do not need them. Qwick-3.5-9B is a fine-tune aimed squarely at that habit, keeping Qwen's answer quality while producing noticeably shorter reasoning.

On the benchmarks below, accuracy stays close to the base model while mean completion length drops by roughly 20 to 55% (see the token-change column).

Evaluation scope: training and model selection used text prompts. A matched BF16 vision-retention measurement over all 900 rows of the MMMU validation split is reported below; it is not the 10,500-row test split. The Hub pipeline remains text-generation because broad multimodal serving was not release-qualified.

Public benchmark results

These are local matched runs against the pinned Qwen3.5-9B checkpoint. Completion length includes thinking and the final answer.

Benchmark	Qwen3.5-9B	Qwick-3.5-9B	Score change	Token change	Profile
MMLU-Pro (12,032)	81.383%	81.117%	-0.266 pp	-39.51%	A
GPQA-Diamond (198)	77.778%	79.798%	+2.020 pp	-28.98%	A
IFEval prompt strict (541)	89.279%	89.649%	+0.370 pp	-22.78%	A
MMMU validation, official-compatible budget (900)	73.778%	74.556%	+0.778 pp	-25.91%	V
JMMLU, common-clean (987)	87.943%	86.930%	-1.013 pp	-54.05%	A
LiveCodeBench v6, budget (1,055)	66.540%	73.839%	+7.299 pp	-51.21%	C

• A: thinking, temperature 1.0, top-p 0.95, top-k 20, presence penalty 1.5, maximum 32,768 new tokens.
• V: same sampling as A, one to five images per prompt, 131,072-token server context, maximum 32,768 new tokens; official-compatible final-content grading with strict no-random-fallback scores co-reported.
• C: thinking, temperature 0.6, top-p 0.95, top-k 20, maximum 81,920 new tokens.

The IFEval rerun uses the same official-recommended general-thinking sampling profile as the other core comparisons. Qwen3.5-9B reached 89.279% and Qwick-3.5-9B reached 89.649% (+0.370 pp), with 8 and 12 length stops. This replaces all earlier temperature-zero IFEval results. IFEval remains report-only and did not trigger reselection.

The vision result covers the complete 900-row validation split (847 multiple-choice and 53 open-answer rows) from MMMU/MMMU@98e6ac0cb9b7b2cd2c991b85a50762edc4aedc68. Qwen3.5-9B scored 664/900 and Qwick-3.5-9B scored 671/900. Qwick minus Qwen3.5-9B is +0.778 pp; the paired subject-stratified bootstrap interval is -1.889 to +3.444 pp and exact McNemar is p=0.6322. The interval crosses zero: this run detects no BF16 vision degradation, but does not resolve an improvement. The official parser used random fallback on 14 / 21 Qwen3.5-9B/Qwick MC parse misses; deterministic strict budget scores were 662/900 and 667/900. Full traces are complete; Qwen3.5-9B/Qwick length stops were 12 / 15. This report-only measurement is not the 10,500-row test split.

The two public quantized checkpoints were subsequently run under the same complete validation protocol:

Model	Official-compatible budget	Strict budget	Mean full-trace tokens	Delta vs Qwick BF16
Qwen3.5-9B BF16	664/900 (73.778%)	662/900 (73.556%)	6476.48	-0.778 pp
Qwick BF16	671/900 (74.556%)	667/900 (74.111%)	4798.66	+0.000 pp
Qwick FP8	655/900 (72.778%)	655/900 (72.778%)	5022.40	-1.778 pp
Qwick NVFP4	635/900 (70.556%)	626/900 (69.556%)	5437.84	-4.000 pp

Qwick BF16, FP8, and NVFP4 use the same model-independent prompt-hash seed per item. FP8 and NVFP4 comparisons to Qwick BF16, their paired descriptive intervals, fallback counts, and trace-integrity details are in EVALUATION.md. These measurements are report-only and do not qualify broad multimodal deployment.

As a Qwen3.5-9B-only budget diagnostic, the 8 Qwen3.5-9B BF16 rows that stopped at 32k were regenerated with a 128k cap. The logical 541-row overlay reached 90.018% prompt strict (+0.739 pp), with length stops and empty final content both falling from 8 to 2. This is not a matched Qwick comparison or a pure token-cap estimate because the feasible runtime/replica layout also changed.

Frozen internal release gate

The 384-prompt synthetic holdout was evaluated once after the candidate weights and its separate gate policy had been frozen. Its predeclared temperature=0 decode is an internal candidate-acceptance condition, not the measurement profile used for the public benchmark table above. The candidate passed that frozen gate. The holdout was not used for training, checkpoint selection, or post-result retuning, and it was not rerun. heldout_results.json records the gate policy, aggregate outcome, and artifact identities; its scores are not comparable to the temperature-1.0 public evaluations.

Quick start

Install a CUDA-compatible PyTorch build, then install Transformers and Accelerate:

python -m pip install -r requirements.txt

import torch
from transformers import AutoTokenizer, Qwen3_5ForConditionalGeneration

model_id = "horiuchinobuyuki/Qwick-3.5-9B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = Qwen3_5ForConditionalGeneration.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
).eval()

inputs = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Explain why the harmonic series diverges."}],
    tokenize=True,
    add_generation_prompt=True,
    enable_thinking=True,
    return_tensors="pt",
    return_dict=True,
).to(next(model.parameters()).device)

with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=8192,
        do_sample=True,
        temperature=1.0,
        top_p=0.95,
        top_k=20,
        min_p=0.0,
        repetition_penalty=1.0,
    )

print(tokenizer.decode(output[0, inputs.input_ids.shape[-1]:], skip_special_tokens=True))

A runnable version is included in examples/text_generation.py. The 8,192-token cap above is convenient for a first run; profile A used vLLM presence_penalty=1.5 and a 32,768-token cap. Qwen3.5 also supports enable_thinking=False, although model selection used thinking mode.

The six weight shards total 18.82 GB (17.53 GiB). A 24 GB GPU is a practical starting point for short BF16 prompts; available context depends on the backend and KV-cache size. CPU offload and quantization can reduce device memory use.

Intended use

The model is intended primarily for research and evaluation of text reasoning, instruction following, Japanese question answering, and code generation. The complete MMMU validation measurement is capability-retention evidence, not broad multimodal deployment qualification. It has not been evaluated for safety-critical advice, unsupervised high-impact decisions, or autonomous action. Validate outputs and benchmark the intended prompt and image distribution before deployment.

How it was made

A rank-8 attention adapter was trained with full-trace SFT and composition-native Step-DPO, then refined by checkpoint search and a reproducible local antithetic NES step. After evaluation, the adapter was merged into the pinned Qwen3.5-9B checkpoint with PEFT safe_merge and serialized as BF16 weights.

The exact Base revision, training-update hash, merge inputs, shard hashes, and evaluation identities are recorded in provenance.json and publication-manifest.json. More detail is in TRAINING.md and DATA.md.

Limitations

• The four-seed public HMMT run regressed from 187/240 to 172/240 budget-correct (-6.25 pp), despite using 32.06% fewer completion tokens.
• The largest MMLU-Pro subject regression was engineering: 733/969 versus 753/969 (-2.064 pp).
• The reserved synthetic holdout used a separately frozen temperature-zero internal release gate; it is not a public performance benchmark and was not rerun or used for retuning.
• The vision result is the complete 900-row validation split with one paired sampling seed per item. Official-parser random-fallback counts and deterministic strict scores are co-reported.
• Reported results apply to this BF16 checkpoint and the listed decoding profiles. The public FP8/NVFP4 MMMU validation results are co-reported, but other converted formats and target image distributions require separate validation.

License and citation

Qwick-3.5-9B is released under Apache-2.0. It is derived from Qwen/Qwen3.5-9B, also published under Apache-2.0. See NOTICE for attribution.

@misc{horiuchi2026qwick35,
  author       = {Nobuyuki Horiuchi},
  title        = {Qwick-3.5-9B},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/horiuchinobuyuki/Qwick-3.5-9B}}
}