Qengineering/Qwen3.5-4B-rk3588

Qwen3.5-4B for RK3588 NPU

This repository provides a hardware-accelerated port of Qwen3.5-4B optimized for Rockchip RK3588 NPU.

User:<image>Describe the image.

Answer: This is a surreal, humorous, and slightly absurd depiction of an astronaut relaxing on the Moon — likely in a fictional or satirical context.

Key elements:

• Astronaut: A person dressed in a bulky white spacesuit with a gold-tinted visor helmet is reclining casually on what appears to be a makeshift chair or cushion made from a green cooler box (possibly repurposed as seating). Their posture suggests complete relaxation — legs stretched out, one hand holding a green glass bottle.

• The Bottle: The astronaut is holding a clear green glass bottle with a label that reads “BrewDog” — a well-known Scottish craft beer brand. This adds to the comedic contrast: enjoying a beer in space? It’s impossible under current physics and regulations, but it fits the whimsical tone.

• Setting: The background shows the lunar surface — gray, cratered terrain with no atmosphere or clouds. In the distance, Earth hangs prominently in the blackness of space, dotted with stars. A ladder leans against a structure (perhaps part of a base or rover) on the right side.

• Tone & Style: The image blends realism (detailed spacesuit, realistic Moon surface and Earth) with fantasy and humor. It plays on pop culture tropes — astronauts drinking beer in space is often used for comedic effect in movies, cartoons, or memes.

Interpretation: This isn’t a real photograph — it’s digitally manipulated or conceptually imagined art meant to evoke amusement. It juxtaposes the solemnity of space exploration with the trivial pleasure of enjoying a drink, creating irony and levity.

In short: An astronaut lounging on the Moon, sipping BrewDog beer, with Earth watching over them. 🌕🍻🌍

---

Model Files

Component	File	Precision
LLM	qwen3.5-4b_w8a8_rk3588.rkllm	W8A8
Vision Encoder	qwen3.5-4b-vision_rk3588.rknn	FP16

Hardware Requirements

• Rockchip RK3588 / RK3588S
• RKNPU2 driver
• Tested on:
  • Rock 5C
  • Ubuntu 22.04 / 24.04 (Joshua Riek)

Runtime Requirements

• RKLLM runtime (version 1.3.0)
• RKNN runtime (rknpu2)
• OpenCV (for image preprocessing)

Model performance benchmark (FPS)

All models, with C++ examples, can be found on the Q-engineering GitHub.

All LLM models are quantized to w8a8, while the VLM vision encoders use fp16.

model	RAM (GB)1	llm cold sec2	llm warm sec3	vlm cold sec2	vlm warm sec3	Resolution	Tokens/s
Qwen3.5-9B	9.2	97.1	97.1	11.5	11.5	448 x 448	3.2
Qwen3.5-4B	5.4	52.8	6.2	8.5	0.9	448 x 448	5.2
Qwen3.5-2B	2.9	23.9	3.2	8.5	0.8	448 x 448	11.0
Qwen3.5-0.8B	1.3	10.6	1.9	2.7	0.2	448 x 448	21.6
Qwen3-2B	3.1	21.9	2.6	10.0	0.9	448 x 448	11.5
Qwen3-4B	8.7	49.6	5.6	10.6	1.1	448 x 448	5.7
InternVL3.5-1B	1.9	8.3	8.0	1.5	0.8	448 x 448	24
InternVL3.5-2B	3.0	22	8.0	2.7	0.8	448 x 448	11.2
InternVL3.5-4B	5.4	50	8.0	5.9	0.8	448 x 448	5
InternVL3.5-8B	8.8	92	8.0	50.5	5.8	448 x 448	3.5
Qwen2.5-3B	4.8	48.3	4.0	17.9	1.8	392 x 392	7.0
Qwen2-7B	8.7	86.6	34.5	37.1	20.7	392 x 392	3.7
Qwen2-2.2B	3.3	29.1	2.5	17.1	1.7	392 x 392	12.5
InternVL3-1B	1.3	6.8	1.1	7.8	0.75	448 x 448	30
SmolVLM2-2.2B	3.4	21.2	2.6	10.5	0.9	384 x 384	11
SmolVLM2-500M	0.8	4.8	0.7	2.5	0.25	384 x 384	31
SmolVLM2-256M	0.5	1.1	0.4	2.5	0.25	384 x 384	54

¹ The total used memory; LLM plus the VLM.
² When an llm/vlm model is loaded for the first time from your disk to RAM or NPU, it is called a cold start.
The duration depends on your OS, I/O transfer rate, and memory mapping.
³ Subsequent loading (warm start) takes advantage of the already mapped data in RAM. Mostly, only a few pointers need to be restored.

Example Usage

• see: https://github.com/Qengineering/Qwen3.5-4B-NPU

Notes

• KV Cache up to 16384
• This is not a Transformers-compatible model
• This repository provides precompiled NPU binaries
• CPU fallback is not supported