Instructions to use wcamon/circus-0.4-t9 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use wcamon/circus-0.4-t9 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="wcamon/circus-0.4-t9")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("wcamon/circus-0.4-t9")
model = AutoModelForMultimodalLM.from_pretrained("wcamon/circus-0.4-t9", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use wcamon/circus-0.4-t9 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "wcamon/circus-0.4-t9"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "wcamon/circus-0.4-t9",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/wcamon/circus-0.4-t9

SGLang

How to use wcamon/circus-0.4-t9 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "wcamon/circus-0.4-t9" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "wcamon/circus-0.4-t9",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "wcamon/circus-0.4-t9" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "wcamon/circus-0.4-t9",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use wcamon/circus-0.4-t9 with Docker Model Runner:
```
docker model run hf.co/wcamon/circus-0.4-t9
```

circus-0.4-t9 — 9-level additive bit-plane QAT recovery of Agents-A1-4B

Author: Wei-Ciao Wu (@wcamon) Code: github.com/wcAmon/guava-qat — full quantization / reconstruction / eval facility that produced this model. Status: stage result (v0.4.1, "t9" = 9-level grid) — ships both the bf16 materialization and the true packed bit-planes (bitplanes_k2_c0.6.npz, bit-exact, see below); addition-only kernels to follow.

Every linear weight of Agents-A1-4B is constrained to the additive two-plane 9-level grid

W = α ⊙ (T₁ + c·T₂),  T₁,T₂ ∈ {−1,0,+1},  c = 0.6,  α: one scale / 32 weights
grid = {0, ±0.4, ±0.6, ±1, ±1.6}   (≈ 3.67 bits/weight, ≈ 4.4× vs bf16)

The two ternary planes admit addition-only matmul kernels (BitNet-style), and 7/5-level variants of the same family cost the same at inference — 9-level is the capability ceiling of the family, single-plane ternary its compression ceiling.

Two artifacts, one model

model.safetensors — the quantized values materialized in bf16, so the model loads with stock transformers exactly like the base model. This is a storage/compatibility format, not a claim of 16-bit information content: every weight lies on its block's 9-level grid.
bitplanes_k2_c0.6.npz — the true quantized artifact (1.7 GB vs 8.5 GB): per target linear, 4-bit plane indices idx=(T₁+1)·3+(T₂+1), two per byte, in the GPTQ column-permuted domain, plus per-block-32 fp32 scales α and the int32 inverse column permutation inv. Scales are kept fp32 so that decoding is bit-exact against model.safetensors:
```
python decode_bitplanes.py     # verified: 200/200 tensors bit-exact
```
Raw pack ≈ 5 bits/weight (4-bit index + fp32 scale); with fp16 scales and entropy coding of the 9-way index the format reaches ≈ 3.67 bits/weight. No addition-only matmul kernel ships yet — until one exists, the bf16 materialization is how you run the model, the bit-plane pack is what the model is.

Non-target weights (embeddings, lm_head, layernorms, the vision stack) remain bf16 in both forms.

Recovery recipe (single MI325X, < 5 GPU-hours total)

GPTQ-style per-layer init with jointly solved block scales.
Sliding-window soft-anneal reconstruction (3 h): windows of 4 layers, stride 2, per-window multi-threshold tanh softening annealed s→30 over 80 % of epochs, explicit STE hard finish for the last 20 %, LoRA r=64 + multiplicative scale modulation as carriers, huber loss against the fp window's own outputs on 512 calibration segments. This puts the ternary assignment T into the gradient loop (22 % of assignments flip) — the step that end-to-end scale-only KD provably could not do in our ablations. Follows the softened-ternarization + sliding-layer reconstruction line of CAT-Q (ICML'26) / SliderQuant (ICLR'26), generalized from ternary to the additive 9-level grid.
Light polish (9 min): 50 steps of scale-only logit KD from the bf16 teacher.

Results (retention vs. bf16 base, same local harness, full test sets)

task	bf16	circus-0.4-t9	retention
gsm8k (1319)	.8006	.7710	96.3 %
mmlu (full)	.7021	.6953	99.0 %
ifeval (541)	.2625	.2514	95.8 %
humaneval (164)	.5732	.5366	93.6 %
arc-challenge	.5444	.5503	101.1 %
hellaswag	.7169	.6927	96.6 %
winogrande	.6875	.6622	96.3 %
boolq	.8593	.8786	102.2 %
piqa	.7709	.7709	100.0 %

All ten tracked tasks retain ≥ 90 % (mbpp excluded from the headline because its train split occurs in the polish corpus). Scores are lm-eval-harness, no chat template, greedy/likelihood defaults; single seed; your numbers may differ under other harnesses.

Honest scope

At ~3.67 bits this is an easier target than pure ternary (CAT-Q) — the point of the release is the additive bit-plane form (addition-only kernels, 9/7/5 same-cost family) and the recovery methodology, not a bit-budget record.
Findings we believe transfer: (i) at this bit budget, the recovery bottleneck is assignment quality, not scales — local fp-anchored reconstruction fixes in hours what end-to-end KD cannot fix at all; (ii) val CE is not a valid intermediate judge for low-bit recovery (our best model is 0.42 nats worse in CE than a strictly weaker checkpoint); (iii) corpus-mix Pareto trade-offs observed during scale-only polishing dissolve once assignments are repaired.
Base model is a VLM; only the language stack was quantized and evaluated.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained("wcamon/circus-0.4-t9", torch_dtype="bfloat16")
tok = AutoTokenizer.from_pretrained("wcamon/circus-0.4-t9")

Citations

CAT-Q (arXiv:2606.26650) · SliderQuant (ICLR 2026) · BitNet b1.58 (arXiv:2402.17764) · ParetoQ (arXiv:2502.02631) · GPTQ (arXiv:2210.17323) · BRECQ (arXiv:2102.05426) · STE (arXiv:1308.3432) · Agents-A1 (arXiv:2606.30616)