iAcloud/gemma4-26B-A4b-it-rbln-tp8

gemma-4-26B-A4b-it — RBLN (ATOM CA25, 8-device) compiled

google/gemma-4-26B-A4b-it (multimodal MoE, image-text-to-text) pre-compiled for Rebellions RBLN-CA25 NPUs with optimum-rbln, ready to serve with vllm-rbln. Compiled artifact only — you need RBLN NPUs and the RBLN SDK to run it.

Hardware & software

NPU	8× RBLN-CA25 (language model: 8 devices, vision tower: 8 devices)
Model	128-expert MoE, 25.8B params (~4B active), 30 layers (5 full-attn + 25 sliding)
Compiler	rebel-compiler==0.11.0
Runtime	optimum-rbln==0.11.0.post1, vllm-rbln==0.11.0, transformers==5.8.1

Compile configuration

Option	Value
max_seq_len	131,072
batch_size	8
decoder_batch_sizes (dynamic batch)	[1, 2, 4, 8]
attn_impl / kvcache_partition_len	flash_attn / 16384
cache_impl / sliding_window	hybrid / 1024
kvcache_num_blocks	48 (≈786k shared KV tokens)
vision max_soft_tokens / image_prefill_chunk_size	280 / 384 (tuned for ≤3 MB images)

kvcache_num_blocks is pinned (not auto) to leave per-device headroom for the vision tower, which shares all 8 devices with the language model.

Serving

vllm serve <this-repo-or-local-path> \
  --max-num-seqs 8 \
  --max-model-len 131072 \
  --enable-auto-tool-choice \
  --tool-call-parser gemma4 \
  --reasoning-parser gemma4

Notes:

• --max-num-seqs 8 is required — vllm-rbln defaults it to 1.
• Do not pass --tensor-parallel-size; the 8-device split is baked into the compiled model (vllm-rbln forces TP=1).
• Reasoning (thinking) is per-request: send chat_template_kwargs={"enable_thinking": true}; reasoning text is returned in message.reasoning_content.

License

Derived from google/gemma-4-26B-A4b-it; use is subject to the Gemma Terms of Use.