Instructions to use norecyc/Bielik-11B-v3.0-Instruct-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use norecyc/Bielik-11B-v3.0-Instruct-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="norecyc/Bielik-11B-v3.0-Instruct-NVFP4")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("norecyc/Bielik-11B-v3.0-Instruct-NVFP4")
model = AutoModelForCausalLM.from_pretrained("norecyc/Bielik-11B-v3.0-Instruct-NVFP4")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use norecyc/Bielik-11B-v3.0-Instruct-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "norecyc/Bielik-11B-v3.0-Instruct-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "norecyc/Bielik-11B-v3.0-Instruct-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/norecyc/Bielik-11B-v3.0-Instruct-NVFP4

SGLang

How to use norecyc/Bielik-11B-v3.0-Instruct-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "norecyc/Bielik-11B-v3.0-Instruct-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "norecyc/Bielik-11B-v3.0-Instruct-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "norecyc/Bielik-11B-v3.0-Instruct-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "norecyc/Bielik-11B-v3.0-Instruct-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use norecyc/Bielik-11B-v3.0-Instruct-NVFP4 with Docker Model Runner:
```
docker model run hf.co/norecyc/Bielik-11B-v3.0-Instruct-NVFP4
```

Bielik-11B-v3.0-Instruct-NVFP4

NVFP4 (4-bit floating-point) weight-only quantization of speakleash/Bielik-11B-v3.0-Instruct, built for fast inference on NVIDIA Blackwell GPUs and designed to pair with DFlash speculative decoding for the fastest Bielik serving setup.

Community quantization. Unofficial build, not an official SpeakLeash release.

What this is


Base model	`speakleash/Bielik-11B-v3.0-Instruct` (Llama arch, 11B)
Method	LLM Compressor one-shot
Scheme	`NVFP4A16` — weight-only (4-bit NVFP4 weights, BF16 activations)
Format	`compressed-tensors` (`nvfp4-pack-quantized`), group size 16, FP8-E4M3 block scales
Not quantized	`lm_head`
Size	~6.3 GB (from ~21 GB BF16)
License	Apache-2.0 (inherited from base)

Hardware

NVFP4 runs natively on NVIDIA Blackwell (GB10 / DGX Spark, B200, RTX 50-series). Built and tested on NVIDIA GB10 (Grace Blackwell).

Usage (vLLM)

Fastest: NVFP4 + DFlash speculative decoding

vllm serve norecyc/Bielik-11B-v3.0-Instruct-NVFP4 \
  --served-model-name bielik-11b \
  --speculative-config '{"method":"dflash","model":"speakleash/Bielik-11B-v3.0-DFlash","num_speculative_tokens":8}' \
  --max-model-len 32768 --gpu-memory-utilization 0.85 \
  --host 0.0.0.0 --port 8000

Plain (no speculative decoding)

vllm serve norecyc/Bielik-11B-v3.0-Instruct-NVFP4 --max-model-len 32768

Chat template

Use the model's native chat template (bundled chat_template.jinja, ChatML-style). Do not use a tool-calling template (e.g. Hermes) for plain chat — it makes the model emit tool calls instead of answering.

Evaluation

Quality vs the BF16 base, measured with lm-evaluation-harness (loglikelihood / multiple-choice — speculative decoding does not affect these scores, so this isolates quantization quality), n=200 per task:

Task	BF16 base	NVFP4 (this model)	Delta
arc_challenge (reasoning)	0.555	0.575	+0.020
belebele_pol (Polish reading)	0.845	0.855	+0.010

Both differences are within statistical noise (~±3% at n=200) — no measurable quality regression from 4-bit quantization. (n=200 rules out degradation beyond ~3-4%; for sub-1% certainty, run full task sets.)

Performance

Single-stream decode throughput on one NVIDIA GB10 (temp=0), decomposing the two speedup levers — NVFP4 quantization x DFlash speculative decoding:

	DFlash off	DFlash on
BF16	9.6 tok/s	25.9 tok/s
NVFP4	33.7 tok/s	73-78 tok/s

NVFP4 alone 3.5x, DFlash alone 2.7x, combined 7.6x over the BF16 / no-speculation baseline.
Sub-multiplicative (2.7 x 3.5 = 9.5x) because the draft stays BF16 and becomes the relatively larger per-step cost once the target is 4-bit.
Optimal num_speculative_tokens = 8; speculative mean acceptance length ~3.2 of 8.

Quantizing the DFlash draft to NVFP4 is not currently loadable in vLLM (the dedicated qwen3_dflash draft loader does not support compressed-tensors weight packing), so the draft is served in BF16.

Quantization recipe

from llmcompressor import oneshot
from llmcompressor.modifiers.quantization import QuantizationModifier

recipe = QuantizationModifier(targets="Linear", scheme="NVFP4A16", ignore=["lm_head"])
oneshot(model="speakleash/Bielik-11B-v3.0-Instruct", recipe=recipe, output_dir="bielik-nvfp4")

Attribution & license

Base: SpeakLeash — speakleash/Bielik-11B-v3.0-Instruct (Apache-2.0).
Speculative draft: speakleash/Bielik-11B-v3.0-DFlash (Apache-2.0).
Released under Apache-2.0, preserving the base model's license and attribution.

Downloads last month: 30

Safetensors

Model size

6B params

Tensor type

F32

BF16

F8_E4M3

Model tree for norecyc/Bielik-11B-v3.0-Instruct-NVFP4

Base model

speakleash/Bielik-11B-v3-Base-20250730

Finetuned

speakleash/Bielik-11B-v3.0-Instruct

Quantized

(17)

this model