Qengineering/Qwen3.5-2B-rk3588

Qwen3.5-2B for RK3588 NPU

This repository provides a hardware-accelerated port of Qwen3.5-2B optimized for Rockchip RK3588 NPU.

User:<image>Describe the image.

Answer: Answer: This is a surreal, digitally manipulated image that blends science fiction with everyday life in an unexpected setting. Here’s a detailed description:

Main Subject:

• A person wearing a white astronaut suit reclines on what appears to be a lunar surface or spacecraft interior.
• Their legs are propped up against the edge of a green cooler box, which has faint graffiti-style writing (“SOMETHING”) visible on its side.
• In their right hand, they hold a bright green glass beer bottle — likely a craft beer — as if enjoying a drink in zero gravity.

Setting & Background:

• The environment is clearly extraterrestrial: gray, cratered terrain resembling the Moon’s surface.
• Behind the astronaut, Earth dominates the sky — its blue oceans and swirling white clouds are visible against the black void of space dotted with stars.
• To the right, part of a metallic ladder or structure leans into the frame, suggesting this might be inside a spacecraft or on a lunar module.

Atmosphere & Style:

• The lighting is dramatic — strong highlights on the astronaut’s helmet visor and suit, contrasting sharply with the dark background.
• There’s a sense of irony or absurdity: an astronaut relaxing with beer next to Earth, defying gravity and normal human behavior in space.
• The image has a hyper-realistic yet dreamlike quality, enhanced by digital effects like lens flares and atmospheric glow around Earth.

Overall Impression: It’s a humorous, thought-provoking piece that plays on the contrast between isolation in space and comfort found in mundane pleasures — drinking beer while floating above our own planet. It invites reflection on human resilience, curiosity, and the strange ways we adapt to extreme environments.

---

Model Files

Component	File	Precision
LLM	qwen3.5-2b_w8a8_rk3588.rkllm	W8A8
Vision Encoder	qwen3.5-2b-vision_rk3588.rknn	FP16

Hardware Requirements

• Rockchip RK3588 / RK3588S
• RKNPU2 driver
• Tested on:
  • Rock 5C
  • Ubuntu 22.04 / 24.04 (Joshua Riek)

Runtime Requirements

• RKLLM runtime (version 1.3.0)
• RKNN runtime (rknpu2)
• OpenCV (for image preprocessing)

Model performance benchmark (FPS)

All models, with C++ examples, can be found on the Q-engineering GitHub.

All LLM models are quantized to w8a8, while the VLM vision encoders use fp16.

model	RAM (GB)1	llm cold sec2	llm warm sec3	vlm cold sec2	vlm warm sec3	Resolution	Tokens/s
Qwen3.5-9B	9.2	97.1	97.1	11.5	11.5	448 x 448	3.2
Qwen3.5-4B	5.4	52.8	6.2	8.5	0.9	448 x 448	5.2
Qwen3.5-2B	2.9	23.9	3.2	8.5	0.8	448 x 448	11.0
Qwen3.5-0.8B	1.3	10.6	1.9	2.7	0.2	448 x 448	21.6
Qwen3-2B	3.1	21.9	2.6	10.0	0.9	448 x 448	11.5
Qwen3-4B	8.7	49.6	5.6	10.6	1.1	448 x 448	5.7
InternVL3.5-1B	1.9	8.3	8.0	1.5	0.8	448 x 448	24
InternVL3.5-2B	3.0	22	8.0	2.7	0.8	448 x 448	11.2
InternVL3.5-4B	5.4	50	8.0	5.9	0.8	448 x 448	5
InternVL3.5-8B	8.8	92	8.0	50.5	5.8	448 x 448	3.5
Qwen2.5-3B	4.8	48.3	4.0	17.9	1.8	392 x 392	7.0
Qwen2-7B	8.7	86.6	34.5	37.1	20.7	392 x 392	3.7
Qwen2-2.2B	3.3	29.1	2.5	17.1	1.7	392 x 392	12.5
InternVL3-1B	1.3	6.8	1.1	7.8	0.75	448 x 448	30
SmolVLM2-2.2B	3.4	21.2	2.6	10.5	0.9	384 x 384	11
SmolVLM2-500M	0.8	4.8	0.7	2.5	0.25	384 x 384	31
SmolVLM2-256M	0.5	1.1	0.4	2.5	0.25	384 x 384	54

¹ The total used memory; LLM plus the VLM.
² When an llm/vlm model is loaded for the first time from your disk to RAM or NPU, it is called a cold start.
The duration depends on your OS, I/O transfer rate, and memory mapping.
³ Subsequent loading (warm start) takes advantage of the already mapped data in RAM. Mostly, only a few pointers need to be restored.

Example Usage

• see: https://github.com/Qengineering/Qwen3.5-2B-NPU

Notes

• KV Cache up to 16384
• This is not a Transformers-compatible model
• This repository provides precompiled NPU binaries
• CPU fallback is not supported