Instructions to use culture-dev/clef-27b-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use culture-dev/clef-27b-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="culture-dev/clef-27b-NVFP4")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("culture-dev/clef-27b-NVFP4")
model = AutoModelForMultimodalLM.from_pretrained("culture-dev/clef-27b-NVFP4", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use culture-dev/clef-27b-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "culture-dev/clef-27b-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "culture-dev/clef-27b-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/culture-dev/clef-27b-NVFP4

SGLang

How to use culture-dev/clef-27b-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "culture-dev/clef-27b-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "culture-dev/clef-27b-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "culture-dev/clef-27b-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "culture-dev/clef-27b-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use culture-dev/clef-27b-NVFP4 with Docker Model Runner:
```
docker model run hf.co/culture-dev/clef-27b-NVFP4
```

Clef 27B NVFP4

An NVFP4 (W4A4) quantization of Cloudflare/clef at revision 2f3de3dd85f379784083b0814d997ab627200f0c, built for NVIDIA Blackwell GPUs (FP4 tensor cores), such as DGX Spark GB10 and Jetson AGX Thor. It does not run efficiently on Ampere or older GPUs; use a W4A16 build there.

This is not an official Cloudflare release. For what Clef is, its API and its evaluations, read the upstream model card.

Evaluation

Measured 2026-10-14 on a 320-record / 418-question held-out probe set (eval splits of boolq, SNLI, banking77, CommonsenseQA, ARC-Challenge, go_emotions, plus synthetic incident and routing records), against Cloudflare's original bf16 weights. Served with vLLM 0.31.0 as a pooling model (FlashInferCutlassNvFp4LinearKernel):

	bf16 original	DGX Spark GB10	Jetson AGX Thor
Top-1 accuracy	93.5%	93.3%	92.1% / 91.9%
Top-1 agreement with bf16	—	97.85%	95.7% / 96.4%
Mean KL from bf16	—	0.007	1.01
Mean top-1 probability	0.92	0.92	0.31

Known issue: Jetson AGX Thor (sm_110). On the Thor, the top answer stays close to bf16 (95.7% and 96.4% agreement, 1.5 to 2.2 points below the Spark's 97.85%), but the probabilities are flattened: the mean top-1 probability is 0.31 instead of 0.92, and the KL from bf16 is about 140 times the Spark's. This holds under both TRITON_ATTN and FLASH_ATTN attention (the two Thor columns), so the cause is something else specific to sm_110, not yet identified. On the Thor, use the top answer only and do not rely on the probabilities.

On the Spark, the median latency per decision is 181 ms for prompts up to 1,024 tokens, about 2.4x faster than bf16.

What changed (Apache-2.0 section 4b)

The full-attention and MLP Linear layers of the backbone (256 layers) are quantized to NVFP4: FP4 weights and activations, block size 16, FP8 E4M3 block scales plus FP32 global scales, stored as compressed-tensors nvfp4-pack-quantized.
Kept in bf16: lm_head, embed_tokens, the vision tower, and every GatedDeltaNet (linear_attn) projection. llm-compressor's own Qwen3.5 NVFP4 recipe documents the fused GatedDeltaNet projections as incompatible with NVFP4. Norms, conv1d, A_log and dt_bias are not Linear layers and are unchanged.
The joint schema head (joint_head.safetensors) and its code (joint_schema_model.py) are copied unchanged from the source.
quantization_provenance.json and recipe.yaml record the exact scheme, calibration and library versions.

Calibration

481 decision records (140,614 tokens), each encoded with Clef's own encode_record, so activation scales were calibrated on the same token sequences Clef scores. Records longer than 4,096 tokens were dropped, not truncated. The set is drawn from public datasets' train splits, pinned by revision, plus synthetic seeds.

Tooling: llm-compressor 0.14.0, compressed-tensors 0.19.0, transformers 5.17.0, torch 2.13.0.

License

Apache-2.0, the same as the source. See LICENSE.

Downloads last month: 15

Safetensors

Model size

19B params

Tensor type

BF16

Model tree for culture-dev/clef-27b-NVFP4

Base model

Qwen/Qwen3.8-27B

Finetuned

Cloudflare/clef

Quantized

(41)

this model