ruisv/bllm-qwen3.5-2b

Qwen3.5-2B — BPU-native build for D-Robotics RDK S100 / S100P

Compiled, quantised (int8 weights / int16 activations), and board-verified build of Qwen/Qwen3.5-2B for D-Robotics' Nash BPU, running entirely through BLLM — an on-device runtime that talks to the BPU directly (hbDNN/hbUCP), with no GPU/CUDA and no cloud round-trip.

This is a compiled BPU binary, not a transformers-loadable checkpoint — closer in kind to a GGUF quantisation than a HF AutoModel. There is no Inference API widget on this page; see "How to run" below.

New here? Start with BLLM (the runtime) and bllm-model-zoo (how this build was made, full acceptance data, what was tried and rejected). This repo is the downloadable artifact; that repo is the method.

Quick facts

Base model	Qwen/Qwen3.5-2B (Apache-2.0)
Architecture	Hybrid: 18 Gated-DeltaNet (linear-attention SSM) layers + 6 full-attention layers
Quantisation	int8 weight (per-channel) / int16 activation (static, data-free)
Target hardware	D-Robotics RDK S100 / S100P (Nash nash-m BPU)
Modality	Image + text (vision tower included)
BPU utilization	100% — CPU_inference_time_cost == 0 on the text decoder
Runtime	BLLM (native, no libxlm)

This architecture (linear-attention SSM + head_dim=256) is not covered by D-Robotics' official OE-LLM toolchain (libxlm) — this build exists because BLLM's native runtime compiles the graph directly.

Files in this repo

s100p/
  ctx4k/    # cache_len=4096 — default, recommended context
    model.hbm            # text decoder (prefill+decode dual-graph)
    model_prefill.hbm     # chunked-prefill graph (N=32)
    model.json
    embed_tokens.bin
    tokenizer.json
    visual.hbm            # vision tower, 320px bucket, 100 tokens/image
  ctx512/   # cache_len=512 — faster decode, short-context use
    ...same layout

cache_len is a performance knob as well as a capacity one — a larger KV window costs more DDR traffic per decoded token. Pick ctx512 if your prompts/replies genuinely fit in ~512 tokens; it decodes faster than ctx4k.

How to run

conda install -c https://mirrors.ruis.ai/conda -c conda-forge bllm

import bllm
m = bllm.load("s100p/ctx4k")
print(m.chat("这张图里有什么?", images=["photo.jpg"]))

# text-only, without loading the vision tower into memory
m = bllm.load("s100p/ctx4k", vision=False)
print(m.chat("介绍一下你自己"))

Requires D-Robotics RDK S100/S100P, sw >= 4.0.5, and an enlarged ION carveout (hb_switch_ion.sh balanced + reboot) — see BLLM's docs/LLM_ONBOARD.md.

Measured on hardware (S100P, performance mode, 3 repeated runs)

variant	decode	TTFT (image, coco-cats test photo)	prefill↔decode parity	ppl	image contrast	task anchors
ctx4096	13.20 tok/s	1.65 s	PASS	51.23	PASS	PASS
ctx512	15.29 tok/s	1.51 s	PASS	50.87	PASS	PASS

Measured against this release's GQA batched-matmul + mask-dedup attention rewrite (replacing an expand_win broadcast) — essentially unchanged from the pre-rewrite baseline (13.22 tok/s), consistent with board output verified bit-identical to that baseline. Full methodology, acceptance protocol (prefill↔decode parity + perplexity + task-coherence anchors + with/without-image contrast — not just cosine similarity, see why in the model-zoo README), and every rejected build are in bllm-model-zoo/models/qwen3.5-2b and .../qwen3.5-vlm.

Known limitations

• The all-BPU chunked-prefill path (image ingestion, no CPU fallback) is numerically solvable but impractically slow to compile — this build's prefill graph uses a float [C,C] CPU fallback for one small intermediate instead. A compile-time/build-engineering tradeoff, not an accuracy or runtime limitation of the shipped model.
• int16 activation quantisation is data-free (no calibration set) at this size — verified sufficient at 2B; the 4B sibling needs per-linear calibration (see its model card).

License

The compiled .hbm is a derivative of the upstream Qwen3.5-2B weights and inherits its Apache-2.0 license — check the upstream model card for the current terms. BLLM (the runtime that loads this file) is open source; see its repository for its own license.