Instructions to use Blackfrost-AI/BLACKFROST-3.8-ICED-BF16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Blackfrost-AI/BLACKFROST-3.8-ICED-BF16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Blackfrost-AI/BLACKFROST-3.8-ICED-BF16")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Blackfrost-AI/BLACKFROST-3.8-ICED-BF16")
model = AutoModelForMultimodalLM.from_pretrained("Blackfrost-AI/BLACKFROST-3.8-ICED-BF16", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Blackfrost-AI/BLACKFROST-3.8-ICED-BF16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Blackfrost-AI/BLACKFROST-3.8-ICED-BF16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Blackfrost-AI/BLACKFROST-3.8-ICED-BF16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Blackfrost-AI/BLACKFROST-3.8-ICED-BF16

SGLang

How to use Blackfrost-AI/BLACKFROST-3.8-ICED-BF16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Blackfrost-AI/BLACKFROST-3.8-ICED-BF16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Blackfrost-AI/BLACKFROST-3.8-ICED-BF16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Blackfrost-AI/BLACKFROST-3.8-ICED-BF16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Blackfrost-AI/BLACKFROST-3.8-ICED-BF16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Blackfrost-AI/BLACKFROST-3.8-ICED-BF16 with Docker Model Runner:
```
docker model run hf.co/Blackfrost-AI/BLACKFROST-3.8-ICED-BF16
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

BLACKFROST-3.8-ICED-BF16

BLACKFROST-3.8-ICED-BF16 is a Blackfrost Research modified BF16 checkpoint using the experimental Qwen4ExpForConditionalGeneration architecture. This is the full-precision speed-trial artifact; compression for smaller single-GPU targets is separate future work.

Checkpoint summary

BF16 safetensors checkpoint: 131 weight shards, approximately 360 GB on the Hub.
48 text blocks with hybrid linear/full attention and a full-attention block every fourth layer.
Hidden size 2,560; 24 attention heads; 2 KV heads.
512 routed experts, top-10 routing, plus a shared expert.
One native hybrid MTP layer is included in the checkpoint.
Configured maximum context: 262,144 tokens. The trial below used 32,768.
The repository contains a vision configuration, but the current validation and benchmark are text-only.

On 2026-09-15, all 131 local weight-shard SHA-256 hashes were checked against the corresponding Hub LFS object IDs with zero mismatches. The current Hub weights are therefore byte-identical to the modified BF16 speed-trial checkpoint.

Native MTP status

The embedded MTP tensors predate the latest trunk-weight modification, so the MTP head should be treated as provisional rather than freshly aligned to the modified model. It nevertheless provides a useful BF16 speculative-decoding baseline. A newly adapted MTP head should be evaluated separately before replacing these tensors.

Current runtime limitations for this hybrid architecture also matter:

Cross-request prefix-cache reuse is disabled by the tested vLLM build when native MTP is combined with the model's Mamba groups.
Fused multi-step draft decode is not yet available for the QWEN4_EXP_EXP_QSA_STATE attention backend, so metadata is rebuilt between speculative draft steps.

Four-B300 BF16 speed trial

Environment: four NVIDIA B300 SXM6 GPUs, tensor parallelism 4, vLLM 0.29.1rc1.dev13+g1cfd97281, 32,768-token serving context, sequential requests, and thinking enabled. The frozen suite contained eight reasoning, code, prose, and tool-result-synthesis prompts with temperature 1.0, top-p 0.95, top-k 20, seed 38421, and at most 256 generated tokens.

Mode	Completion tokens	Median decode	Mean decode	Median TTFT	Draft acceptance	Relative median
No draft	1,760	190.10 tok/s	190.51 tok/s	0.696 s	n/a	baseline
Native MTP, k=2	1,820	265.00 tok/s	264.27 tok/s	0.660 s	55.0%	+39.4%
Native MTP, k=3	1,799	263.10 tok/s	288.43 tok/s	0.684 s	49.6%	+38.4%

Per-position acceptance was 65.7% / 44.4% for k=2 and 66.8% / 47.8% / 34.3% for k=3. The k=3 mean was lifted by two unusually fast cases, while its median did not beat k=2. Native MTP with k=2 is therefore the current BF16 trial setting.

These are single-pass development measurements, not a production throughput claim. Workload, concurrency, context length, runtime build, and sampling settings can materially change the result.

Example vLLM launch

The benchmarked profile used exactly GPUs 0-3:

CUDA_VISIBLE_DEVICES=0,1,2,3 \
VLLM_WORKER_MULTIPROC_METHOD=spawn \
vllm serve Blackfrost-AI/BLACKFROST-3.8-ICED-BF16 \
  --served-model-name BLACKFROST-3.8-ICED-BF16 \
  --tensor-parallel-size 4 \
  --max-model-len 32768 \
  --max-num-seqs 4 \
  --gpu-memory-utilization 0.45 \
  --enable-prefix-caching \
  --trust-remote-code \
  --reasoning-parser qwen3 \
  --mamba-ssm-cache-dtype bfloat16 \
  --speculative-config.method mtp \
  --speculative-config.num-speculative-tokens 2

The model config specifies a float32 Mamba SSM cache. The command above deliberately overrides that cache to BF16 for this speed/memory trial; omit the override when evaluating the configured cache precision.

License

Use of this checkpoint is governed by the Qwen Community License 1.0. Review the license terms before use or redistribution.

Downloads last month: 7

Safetensors

Model size

180B params

Tensor type

BF16

I64

Model tree for Blackfrost-AI/BLACKFROST-3.8-ICED-BF16

Quantizations

2 models