Qwen3-4B-Instruct - Renesas X5H

Introduction

This repository contains Qwen3-4B-Instruct model, optimized for Renesas X5H platform for Image inference.

• Model Architecture: Qwen3‑4B is a lightweight decoder‑only unimodal Transformer that jointly processes visual and textual inputs in an auto‑regressive framework using an integrated vision encoder and efficient attention mechanisms.

• Source Model: Qwen/Qwen3-4B-Instruct

Performance

The following performance metrics were measured with a prompt.

Model	Precision	Device	Prefill (tok/s)	token-gen (tok/sec)
Qwen3-4B-Instruct	W4A16	X5H - Single Cluster NPX	618	19.3

Prerequisites

To run model, you need:

1. Renesas X5H Board
2. Runner installer
3. Hugging Face CLI: For downloading the model.

Qwen3-4B-Instruct (W4A16)

1. Download the installer npu-llm-openai-server-0.1.3-Linux.sh from Files and version tab under w4a16\binaries\rcar-x5hv1\xOS_v4.32 folder.
2. Copy the installer to the X5H board and run the installer.

   bash ./npu-llm-openai-server-0.1.3-Linux.sh --prefix=./ --exclude-subdir --skip-license
   

3. Expected directory structure on the X5H board.

   npu-llm-openai-server
    ├── firmwares
    ├── kernel_modules
    ├── npu-llm-openai-server
    ├── qwen3-4b-llm-w4a16-pkg
    ├── scripts
    ├── setup_npu.sh
    └── tools
   

Inference

Qwen3-4B-Instruct

The NPU LLM OpenAI Server uses a client-server architecture and provides an OpenAI-compatible REST API for inference requests.

1. Start the Server

   bash ./setup_npu.sh
   ./npu-llm-openai-server
   

2. Send an Inference Request
   1. Open a new terminal and connect to the target board using SSH

      ssh root@<board-ip>
      

   2. Navigate to the tools directory

      cd <installed-dir>/npu-llm-openai-server/tools
      

   3. Send a VLM request using send_request.sh

      bash send_request.sh "<prompt>"