NPU2/Q4NX
Collection
These work with FastFlowLM • 21 items • Updated
Configuration Parsing Warning:Config file tokenizer_config.json cannot be fetched (too big)
Configuration Parsing Warning:In config.json: "num_experts" must be a number
IF YOU USE COMMUNITY QWEN MODELS DO NOT UPGRADE TO FLM v1.0.2+
Gemma4-E4B-GLM-FLASH - Q4NX for FastFlowLM (AMD Ryzen AI XDNA2)
A Google Gemma-4-E4B-IT fine-tune (GLM-FLASH style), converted to Q4NX for FastFlowLM.
What is Q4NX?
Q4NX is FastFlowLM's native packed-quantization format - a rearranged Q4_1 layout tuned for the NPU matrix engine's tile sizes and memory access patterns. It is not a GGUF file and it does not run on llama.cpp or Ollama; it is meant exclusively for the FastFlowLM engine on AMD Ryzen AI NPUs.
Requirements
- FastFlowLM >= 0.9.45 (
flmCLI) - AMD Ryzen AI processor with XDNA2 (NPU2) - Strix Point / Ryzen AI 300 series or later
- Linux with the XRT NPU stack installed
- ~15 GB of unified system memory (Q4NX weights + activations + KV cache)
Files
| File | Purpose |
|---|---|
| model.q4nx | Quantized Q4NX weights |
| config.json | FastFlowLM model configuration |
| tokenizer.json | Tokenizer |
| tokenizer_config.json | Special tokens and chat template |
| chat_template.jinja | Chat template (optional) |
| vision_weight.q4nx | Vision tower weights (multimodal input) |
| audio_weight.q4nx | Audio tower weights |
| flm-add.py | Installer script - registers this model with FastFlowLM |
Install and run
This repository works with flm-add, a small installer that copies the model
into the FastFlowLM user directory and registers the tag. It never
modifies the system FastFlowLM install.
pip install flm-add or uv tool install flm-add
uv tool install flm-add
flm-add Atomic-Germ/Gemma4-E4B-GLM-FLASH-NPU2 --tag gemma4-glm-flash:4b --family gemma4
FLM_CONFIG_PATH="$HOME/.config/flm/model_list.json" FLM_XCLBIN_PATH="$HOME/.config/flm" flm run gemma4-glm-flash:4b
Kernels
FastFlowLM's NPU kernels (xclbins) are closed source and are not shipped in
this repository. flm-add.py links the kernels of the official gemma4-it:e4b
model (Gemma4-E4B-IT-NPU2), because this model shares the same engine family
(gemma4e) and architecture.
Model
- Registry tag:
gemma4-glm-flash:4b - Engine family:
gemma4e - Kernel source: Gemma4-E4B-IT-NPU2
- Context length: 131,072 tokens (from config)
model.q4nxsize: 7.14 GB- Base model: google/gemma-4-E4B-it
- License: apache-2.0
GhostWriter Influence Test (Arbitrary but repeatable benchmark)
Tested on an AMD Ryzen AI 340 Framework 13 laptop.
| Metric | Value |
|---|---|
| Prompt Tokens | 9,210 |
| Completion Tokens | 1,721 |
| Total Tokens | 10,986 |
| Active KV Tokens | 10,986 |
| Max KV Token Capacity | 32,768 |
| KV Token Occupancy | 33.53% |
| Load Duration | 0.000000742 seconds |
| Prefill Duration (TTFT) | 22.25 ms |
| Decoding Duration | 216.65 ms |
| Prefill Speed | 416.54 tokens/sec |
| Decoding Speed | 7.94 tokens/sec |
Original model card
See the upstream model card for training details, benchmarks, and upstream usage. This repository only contains the Q4NX conversion for FastFlowLM.
- Upstream card: google/gemma-4-E4B-it
- Downloads last month
- 1,827