bpx03/Qwen3.8-Flash-Next-QEngine

Qwen3.8-Flash-Next for QEngine (preview)

Qwen3.8-Flash-Next in the weight layout of QEngine, a source-available inference engine for 3× NVIDIA RTX 3090. Download it and run it with QEngine directly, with no conversion step.

Preview. The Flash-Next engine of QEngine is a pre-release. These files only work with QEngine, not with Transformers, vLLM or llama.cpp.

What's inside

• Converted from Intel/Qwen3.8-Flash-Next-W4A16-AutoRound, an INT4 quantization of Qwen/Qwen3.8-Flash-Next.
• base.safetensors, layer-00 … layer-47: the model, regrouped per layer for QEngine (W4A16, mixed precision):
  • routed experts: INT4 from the AutoRound checkpoint, repacked bit-exact;
  • shared expert, DeltaNet projections (in_proj_qkv / z / out_proj), n-gram key/value projections: INT4 (RTN, group 128);
  • QSA attention q/k/v/o and lm_head: BF16 in the files, INT8 in the engine (converted at load);
  • hyper-connections, router, embeddings, indexer, gates, norms: BF16.
• mtp.safetensors / mtp-int4.safetensors: the multi-token-prediction head used for speculative decoding.
• ngram_int8.safetensors: the n-gram embedding table in INT8 (51.8 GB instead of 102.4 GB in BF16). In QEngine's tests its next-token KL against the BF16 table was below the engine's own run-to-run noise.
• expert_prior.bin, cold_experts.json, token_rank.bin: routing statistics and the draft vocabulary that QEngine uses.

Requirements

• 3× RTX 3090 (24 GB) with GPU-to-GPU P2P.
• Host RAM, default mode: the INT8 n-gram table (about 52 GB) is pinned in host memory. That is about 64 GB in total with the conversation archive off (FLASH_KV_ARCHIVE_GB=0), about 80 GB with FLASH_KV_ARCHIVE_GB=16 and about 110 GB with the default 48 GB archive.
• Low-RAM hosts: disk mode (FLASH_NGRAM_PIN=0, NVIDIA open kernel modules). The table stays on disk and the rows each step needs are read through the page cache. Measured on an NVMe SSD with the whole server limited to 12 GB of RAM: short single-user requests about 2× slower (127 vs 262 tok/s) and the median time to first token 1.7 s instead of 0.22 s; with the table in the page cache the speed is close to the pinned mode (250 vs 262 tok/s). Details in the QEngine README.
• About 120 GB of disk.

License

The weights are a derivative of Qwen3.8-Flash-Next and are distributed under the Qwen Community License 1.0 (copy included). QEngine itself has its own license, see its repository.