Qwen3.5-0.8B-INT-W8A8

INT8 W8A8 quantization of Qwen/Qwen3.5-0.8B, produced for development purposes with the simplest possible method: round-to-nearest (RTN) weights — no GPTQ, no calibration data.

Quantization scheme (compressed-tensors W8A8 preset)

	bits	type	strategy	scales
weights	int8	symmetric	per-channel	static, RTN via memoryless_minmax
activations	int8	symmetric	per-token	dynamic (computed at runtime)

Ignored (kept in bf16): ['lm_head', 're:model\\.visual\\..*'] — the vision tower stays full-precision; lm_head is tied to the input embeddings.

Provenance

• Script: pytorch_runtime/scripts/quantize_w8a8.py (dram-computing repo)
• Date: 2026-08-07
• llm-compressor 0.12.0.1, compressed-tensors 0.17.1, transformers 5.10.1, torch 2.12.0+cpu

Usage

from vllm import LLM

llm = LLM("casperhansen/Qwen3.5-0.8B-INT-W8A8")  # compressed-tensors backend auto-detected