Instructions to use JANGQ-AI/dots3-note-prev-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use JANGQ-AI/dots3-note-prev-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="JANGQ-AI/dots3-note-prev-NVFP4")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("JANGQ-AI/dots3-note-prev-NVFP4", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use JANGQ-AI/dots3-note-prev-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "JANGQ-AI/dots3-note-prev-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JANGQ-AI/dots3-note-prev-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/JANGQ-AI/dots3-note-prev-NVFP4

SGLang

How to use JANGQ-AI/dots3-note-prev-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "JANGQ-AI/dots3-note-prev-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JANGQ-AI/dots3-note-prev-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "JANGQ-AI/dots3-note-prev-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JANGQ-AI/dots3-note-prev-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use JANGQ-AI/dots3-note-prev-NVFP4 with Docker Model Runner:
```
docker model run hf.co/JANGQ-AI/dots3-note-prev-NVFP4
```

dots3-note Preview — NVFP4 (JANGQ-AI)

280B-A16B omni MoE · NVFP4 (E2M1) experts · Blackwell / GB10 · TP2

Runtime: available. A complete, self-contained vLLM runtime for 2× DGX Spark (GB10, sm_121) is published and validated — see Runtime. GitHub: https://github.com/jjang-ai/dots3-note-gb10-vllm · Image: ghcr.io/jjang-ai/dots3-note-gb10-vllm:sm121-cu130.

What this is

An NVFP4 requantization of dots-studio/dots3-note-prev, targeted at NVIDIA Blackwell (RTX 50, GB10 / DGX Spark) via vLLM's native modelopt_fp4 path.

Routed experts → NVFP4 (E2M1, group-16, FP8-E4M3 block scales + FP32 global scale — modelopt format).
Kept high-precision (BF16): MLA/DSA attention, shared expert, router gates, the dense layer 0, the MTP / NEXTN draft head (layer 46), and the full vision (MoE-ViT) + audio encoders. Multimodal and speculative decoding are preserved, not stripped.
hf_quant_config.json: quant_algo=NVFP4, 10 exclude patterns (attention / shared / dense-0 / MTP / vision / audio / lm_head).
~174 GB on disk. Fits TP2 across two 121 GB GB10 nodes.

Property	Value
Base	dots-studio/dots3-note-prev (Apache-2.0)
Total / active params	280B / 16B
Experts	256 routed + 1 shared, top-8
Attention	13 DSA (top-2048) + 33 SWA
MTP / DSpark	1 shared NEXTN head, 1.13B (layer 46) — kept BF16
Vision / Audio	MoE-ViT 7B (1.2B act) / dense 800M — kept BF16
Context	up to 512K (DSA-native `fp8_ds_mla` paged KV)
Expert precision	NVFP4 (E2M1, g16)
Input / Output	text · image · video · audio / text

Recommended generation config

Defaults tuned for agentic / tool-use / coding work (not plain instruct chat).

from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:8001/v1", api_key="EMPTY")

resp = client.chat.completions.create(
    model="dots3-note",
    messages=[{"role": "user", "content": "Refactor this module and add tests."}],
    temperature=1.0,          # upstream-recommended sampling
    top_p=0.95,
    max_tokens=8192,          # give agentic/coding turns room
    extra_body={
        # Reasoning control (dots3 is a boolean thinking toggle, not tiered effort):
        #   enable_thinking=True  → deliberate multi-step reasoning (hard reasoning / agent planning)
        #   enable_thinking=False → direct answer (low latency, simple tools)
        "chat_template_kwargs": {"enable_thinking": True},
    },
)

Setting	Recommended	Notes
`temperature`	1.0	upstream default; lower (0.2–0.6) for deterministic code edits
`top_p`	0.95
`enable_thinking`	True for agents/hard reasoning, False for simple/latency-sensitive	dots3 has no separate low/med/high tiers — this boolean is the reasoning switch
`max_tokens`	≥4096 (8192 for coding)	thinking + tool loops need headroom
repetition penalty	none	not recommended for this model

Agentic tool calling is native. When serving, enable --enable-auto-tool-choice --tool-call-parser dots; the chat template emits/parses <tool_call> blocks. Pass your tools=[...] schema as usual — no custom system prompt needed.

Multimodal (image · video · audio)

messages = [{"role": "user", "content": [
    {"type": "video_url", "video_url": {"url": "https://.../clip.mp4"}},
    {"type": "text", "text": "Describe the performance and what can be heard."},
]}]

Video inputs carry their audio track — the model reasons over both jointly.
Image / audio / video all route through the preserved (BF16) encoders; multimodal quality is unaffected by NVFP4 expert quantization.
The preprocessor_config.json for vision/video/audio is included.

Runtime — available

A complete, self-contained vLLM runtime for 2× DGX Spark (GB10, sm_121, CUDA 13, aarch64) over TP2:

GitHub: https://github.com/jjang-ai/dots3-note-gb10-vllm
Docker: ghcr.io/jjang-ai/dots3-note-gb10-vllm:sm121-cu130

DeepSeek Sparse Attention (DSA) is not officially supported on consumer / GB10 Blackwell (sm_120/sm_121) in upstream vLLM — the fused sparse-attention and top-k CUDA kernels are Hopper / datacenter-Blackwell only and crash or emit garbage on GB10. This runtime carries the patches that make it correct and fast:

DSA sparse-MLA decode + indexer → exact torch references (the deep_gemm / FlashInfer sm120 kernels are numerically wrong on GB10 across a KV-block boundary).
DSA top-k → torch.topk on sm_121 (the fused kernel needs ≥128 KB shared memory; GB10 exposes ~99 KB, so it crashes for any large max_model_len). Mirrors upstream vLLM PR #49897.
NVFP4 via Marlin W4A16 (the FlashInfer / CUTLASS FP4 GEMMs no-op on sm_121).
CUDA graphs (FULL_AND_PIECEWISE) for ~1.8–2× decode; NCCL 2.30.4 preload to fix a dual-Spark deadlock; --skip-mm-profiling so multimodal and a large KV pool coexist.

Full root-cause writeup: docs/GB10_FIXES.md.

Validated performance (2× GB10, driver 580.x, TP2)

Metric	Value
Decode (MTP + CUDA graphs)	~24–25 tok/s (vs ~14 eager)
Prefill	>1000 tok/s past ~330k ctx (prefix reuse); decode ~flat in context length
Max coherent context	428,671 tokens — single conversation, 0 errors (KV-pool bound)
Multimodal	image · video · audio all verified
Reasoning	default on; `enable_thinking` toggle per request

Serve (Docker, both nodes)

docker pull ghcr.io/jjang-ai/dots3-note-gb10-vllm:sm121-cu130
# worker on spark2 FIRST, then head on spark1 (host networking; set your node IPs + RoCE iface)
IMAGE=ghcr.io/jjang-ai/dots3-note-gb10-vllm:sm121-cu130 MODEL_DIR=/path/to/dots3-note-prev-NVFP4 \
  HEAD_IP=10.99.0.1 WORKER_IP=10.99.0.2 bash docker/run_worker.sh   # spark2
IMAGE=ghcr.io/jjang-ai/dots3-note-gb10-vllm:sm121-cu130 MODEL_DIR=/path/to/dots3-note-prev-NVFP4 \
  HEAD_IP=10.99.0.1 WORKER_IP=10.99.0.2 bash docker/run_head.sh      # spark1

The container defaults to the fast config (MTP + CUDA graphs + NCCL preload). Every env knob and the from-source patch (patches/apply.sh, no image needed) are documented in the GitHub repo.

Access

Open weights — no gating. Apache-2.0, inherited from upstream dots-studio/dots3-note-prev.

Credits

Base model: dots-studio — dots3-note-prev.
NVFP4 requantization + GB10/TP2 runtime: JANGQ-AI.

Downloads last month: -

Safetensors

Model size

170B params

Tensor type

BF16

F8_E4M3

F32