Instructions to use tcclaviger/Tess-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tcclaviger/Tess-FP8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="tcclaviger/Tess-FP8")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("tcclaviger/Tess-FP8")
model = AutoModelForMultimodalLM.from_pretrained("tcclaviger/Tess-FP8")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tcclaviger/Tess-FP8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tcclaviger/Tess-FP8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tcclaviger/Tess-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/tcclaviger/Tess-FP8

SGLang

How to use tcclaviger/Tess-FP8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tcclaviger/Tess-FP8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tcclaviger/Tess-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tcclaviger/Tess-FP8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tcclaviger/Tess-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use tcclaviger/Tess-FP8 with Docker Model Runner:
```
docker model run hf.co/tcclaviger/Tess-FP8
```

W8A8 block-128 FP8 quant of migtissera/Tess-4-27B

Runtime: standard vLLM fp8 quant_method (e4m3, weight_block_size [128, 128], dynamic activations). All evaluation below was run on tcclaviger/vllm:latest on RDNA 4 (gfx12xx). Not validated on any other hardware at this time.

Tess-FP8

Block-wise W8A8 FP8 (e4m3fn) quantization of Tess-4-27B by Migel Tissera — an agentic, thinking-native finetune of Qwen3.6-27B. Sibling quants: Tess-27B-RFI (int8+4-bit hybrid) and Tess-27B-RFA (all-attention 4-bit). All credit for the model to its author; this repo only changes the numerics.

Quantization by component

All attention and MLP linear weights (self_attn q/k/v/o, MLP gate/up/down, GDN linear-attention in_proj_qkv / in_proj_z / out_proj) — FP8 e4m3fn, block-wise 128×128, per-block bf16 dequant scales (weight_scale_inv), dynamic activation scheme (W8A8).
MTP speculative-decode head — its MLP and attention projections are FP8 by the same scheme; its fc and norms stay bf16.
Kept in bf16 (not quantized) — vision encoder, GDN A_log/conv1d/dt_bias/in_proj_a/in_proj_b, embeddings, norms, and the lm_head.

Tensor layout and naming match the Qwen3.6-27B-FP8 production reference exactly.

Serving context — 512K via YaRN

All evaluation below was run while serving at --max-model-len 524288 (512K tokens), extended from the native 256K window with YaRN via --hf-overrides:

{"text_config": {"rope_parameters": {"rope_type": "yarn", "factor": 2.0,
 "original_max_position_embeddings": 262144, "mrope_interleaved": true,
 "mrope_section": [11, 11, 10], "partial_rotary_factor": 0.25,
 "rope_theta": 10000000}}}

Evaluation results

Six builds measured with identical methodology, each against its own live vLLM endpoint (July 2026): Qwen3.6-27B (bf16 base) → Tess-4-27B (the tune, bf16) → Tess-27B-RFI (int8+4-bit hybrid) → Tess-27B-RFA (all-4-bit) → Tess-FP8 (this build, W8A8 block-128 FP8), with Qwen3.6-35B-A3B (MoE, bf16) as a comparative reference. Bold marks the best score in each row (ties all bolded).

Quantization cost (W8A8 FP8 block-128) — Tess-4-27B → Tess-FP8

Metric	Change
Checkpoint size	55.6 → 29.0 GB (−48%)
WikiText-2 perplexity	6.669 → 6.663 (≈ flat)
Codeneedle overall recall	97.7% → 97.9% (≈ flat)
MC accuracy (4 tasks)	≈ flat (−0.4 to +0.3 pp)
Tool-eval / GSM8K / MMLU / IFEval	87 / 98% / 76% / 90%
Decode @ conc 1 (ISL 128)	59.9 → 86.0 tok/s (+44%)
Decode @ conc 50 (ISL 128)	528 → 560 tok/s (+6%)

Quality

Metric	Qwen3.6-27B (base)	Tess-4-27B	Tess-27B-RFI	Tess-27B-RFA	Tess-FP8	Qwen3.6-35B-A3B
WikiText-2 PPL (n_ctx 2048, lower is better)	7.0559	6.6691	6.6632	6.6292	6.6627	6.5092
ARC-Challenge (acc_norm)	59.30%	60.84%	60.41%	60.32%	60.49%	55.20%
ARC-Easy (acc_norm)	75.93%	77.53%	77.40%	78.87%	77.82%	71.13%
Winogrande (acc)	77.51%	77.43%	77.51%	76.80%	77.66%	73.40%
HellaSwag (acc_norm)	84.12%	84.21%	84.27%	84.05%	84.13%	82.95%

Multiple-choice accuracy is lm-eval loglikelihood scoring, 0-shot.

Long-context positional recall (codeneedle)

Verbatim function recall under 10K–80K-token contexts.

Corpus	Qwen3.6-27B (base)	Tess-4-27B	Tess-27B-RFI	Tess-27B-RFA	Tess-FP8	Qwen3.6-35B-A3B
Python	100%	100%	100%	100%	99.55%	99.09%
C++	98.12%	98.12%	98.44%	98.75%	98.75%	98.44%
Rust	99.69%	99.69%	99.69%	99.69%	99.69%	99.38%
JS (~80K tokens)	93.44%	93.13%	93.75%	93.75%	93.44%	92.19%
Tools	98.26%	99.57%	99.57%	99.57%	99.57%	93.48%
Overall recall	97.81%	97.73%	97.97%	98.05%	97.86%	97.28%

Tool calling & accuracy benches

Bench	Qwen3.6-27B (base)	Tess-4-27B	Tess-27B-RFI	Tess-27B-RFA	Tess-FP8	Qwen3.6-35B-A3B
tool-eval final (full 69, TC-61 excl)	86	85	87	86	87	90
GSM8K (50q)	98.0%	94.0%	98.0%	98.0%	98.0%	96.0%
MMLU (50q)	74.0%	76.0%	82.0%	80.0%	76.0%	64.0%
IFEval (20 prompts, prompt-level)	90.0%	90.0%	90.0%	95.0%	90.0%	90.0%

Decode throughput — tok/s output (ISL 128 / ISL 512)

vllm bench serve, random dataset, OSL 128, saturation, 4× R9700 (gfx1201), TP 4.

Concurrency	Qwen3.6-27B (base)	Tess-4-27B	Tess-27B-RFI	Tess-27B-RFA	Tess-FP8	Qwen3.6-35B-A3B
1	57.0 / 61.4	59.9 / 63.8	75.3 / 69.3	67.9 / 58.2	86.0 / 85.1	91.9 / 114.4
10	304.5 / 233.6	289.6 / 227.8	281.5 / 219.2	292.2 / 194.8	280.6 / 269.2	434.9 / 440.3
25	424.4 / 321.7	492.0 / 341.1	429.1 / 284.0	369.0 / 260.3	533.2 / 355.4	688.9 / 563.7
50	556.0 / 349.8	527.9 / 345.6	452.8 / 295.0	422.6 / 278.4	560.3 / 394.1	889.8 / 702.9

MTP draft acceptance by work category

Measured from live serving logs, k=5 draft tokens, drafted-token-weighted aggregation.

Work category	Overall acceptance	Pos 1	Pos 2	Pos 3	Pos 4	Pos 5
JSON generation	96.7%	99.4%	98.8%	98.0%	94.9%	92.5%
Math	79.9%	96.2%	88.5%	79.9%	70.9%	64.2%
Code	63.8%	89.5%	74.4%	62.0%	51.2%	42.0%
Creative English	62.1%	88.7%	72.9%	59.8%	48.5%	40.7%

Notes

All builds serve on the tcclaviger/vllm:latest image, which has kernel tunes baked in. TunableOp is untuned — GEMMs run on default heuristic-determined values. Base Qwen3.6-27B figures are the 2026-07-12 re-measurement on the same tcclaviger/vllm:latest image and thinking-OFF methodology as every other build, replacing an earlier non-comparable run.

Credits

Tess-4-27B by Migel Tissera (migtissera/Tess-4-27B) — the model quantized here:

@misc{tissera2026tess4,
  title        = {Tess-4-27B},
  author       = {Migel Tissera},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/migtissera/Tess-4-27B}}
}

codeneedle (positional recall) originally by Alexander Ziskind, expanded test suite by tcclaviger (Rob Smith).
Tool-calling scenarios (incl. TC-61) run on tool-eval-bench by SeraphimSerapis (Tim Messerschmidt), scenario methodology adapted from ToolCall-15 by stevibe.
GSM8K, MMLU, and IFEval run via tool-eval-bench's built-in accuracy benchmarks at their defaults: GSM8K 8-shot CoT, MMLU 5-shot, IFEval zero-shot.

Downloads last month: 9

Safetensors

Model size

28B params

Tensor type

BF16

F8_E4M3

Model tree for tcclaviger/Tess-FP8

Base model

Qwen/Qwen3.6-27B

Finetuned

migtissera/Tess-4-27B

Quantized

(32)

this model