Instructions to use JasonW2025/Qwen3.8-Flash-Next-NVFP4-Calibrated-KV with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use JasonW2025/Qwen3.8-Flash-Next-NVFP4-Calibrated-KV with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="JasonW2025/Qwen3.8-Flash-Next-NVFP4-Calibrated-KV")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("JasonW2025/Qwen3.8-Flash-Next-NVFP4-Calibrated-KV")
model = AutoModelForMultimodalLM.from_pretrained("JasonW2025/Qwen3.8-Flash-Next-NVFP4-Calibrated-KV", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use JasonW2025/Qwen3.8-Flash-Next-NVFP4-Calibrated-KV with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "JasonW2025/Qwen3.8-Flash-Next-NVFP4-Calibrated-KV"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JasonW2025/Qwen3.8-Flash-Next-NVFP4-Calibrated-KV",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/JasonW2025/Qwen3.8-Flash-Next-NVFP4-Calibrated-KV

SGLang

How to use JasonW2025/Qwen3.8-Flash-Next-NVFP4-Calibrated-KV with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "JasonW2025/Qwen3.8-Flash-Next-NVFP4-Calibrated-KV" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JasonW2025/Qwen3.8-Flash-Next-NVFP4-Calibrated-KV",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "JasonW2025/Qwen3.8-Flash-Next-NVFP4-Calibrated-KV" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JasonW2025/Qwen3.8-Flash-Next-NVFP4-Calibrated-KV",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use JasonW2025/Qwen3.8-Flash-Next-NVFP4-Calibrated-KV with Docker Model Runner:
```
docker model run hf.co/JasonW2025/Qwen3.8-Flash-Next-NVFP4-Calibrated-KV
```

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Qwen3.8 Flash-Next NVFP4 with calibrated FP8 KV (Spark candidate)

This is a manually gated evaluation release of the NVIDIA-derived nvidia/Qwen3.8-Flash-Next-NVFP4 checkpoint. Every access request must be reviewed by the repo owner. The repository is not a public production promotion.

What is in this release

Native NVIDIA/ModelOpt mixed-precision weights: routed experts use NVFP4 (group size 16), the MTP expert path uses FP8, and the PLE n-gram embedding uses FP8. Protected model regions retain their source precision.
Calibrated FP8 KV metadata: 26 FP32 scales covering 13 main/MTP QSA owners.
vLLM serving recipe tested on DGX Spark GB10: eager execution, MTP3, prefix caching, 262,144-token maximum length, and an 8,192-token output reserve.
CPU-deduplicated PLE reader selected for the qualified private deployment. GPU-deduplication was evaluated and found to have no benefits.

Evidence and limits

The candidate loaded and served privately on Spark-01. Against the original reader (Tony) , the same weights and runtime measured approximately 14% lower mixed-C4 p95 latency. The authored campaign screens were retained (50/60 coding cases, 9/12 task cases, 10/12 matrix cases, and 3/3 image cases), and the campaign exercised the 262,144-token envelope.

This release does not claim a served BF16-teacher KL gate. The mapped GPU PLE path was implemented and measured, but its bounded serving comparison did not show a reliable end-to-end speed gain over CPU dedup.

Use the exact vLLM/runtime compatibility documented. Do not infer support for other GPUs or serving stacks from this artifact.

Provenance

NVIDIA source revision: fc694b54fb0174e0913e6adf86691ef85a4ead47
vLLM source pin: 8a728663c1c3eeace834a95f5654fa653cc1998c
Candidate manifest SHA-256: fcbce8545cff454027e98450a60ce0f139e8cad19ac4f127d2a085ce669ecaf2
Checkpoint manifest SHA-256: 981d8b768a08a0a823243f149d57991bf702ccf4a8b2e876c899c9001a22fb75
Calibration receipt SHA-256: 3bc5c7d188b87f86ae1011b6109786bc82580da926088fcff7e640073f3044fd

The full source, receipts, calibration-source audit, and human reports remain in the accompanying Windows/NUC release bundle.

The upstream NVIDIA model card declares nvidia-open-model-license; the underlying Qwen model declares qwen-community-1.0. Review and comply with both upstream licenses before requesting access.

Downloads last month: -

Safetensors

Model size

120B params

Tensor type

BF16

I64

F8_E4M3

Model tree for JasonW2025/Qwen3.8-Flash-Next-NVFP4-Calibrated-KV

Base model

Qwen/Qwen3.8-Flash-Next

Quantized

nvidia/Qwen3.8-Flash-Next-NVFP4

Quantized

(3)

this model