Startup issues with vllm-openai:qwen38-flash-next

#2
by 1anH - opened

Has anyone been able to get this to startup ok with the vllm-openai:qwen38-flash-next docker image?

ValueError: There is no module or parameter named 'ngram_embedding.weight_scale' in Qwen3_8FlashNextNGramEmbedding. The available parameters belonging to ngram_embedding (VocabParallelEmbedding) are: {'ngram_embedding.weight'}

  --model RadixArk/Qwen3.8-Flash-Next-NVFP4
  --max-num-seqs 12
  --tensor-parallel-size 2
  --gpu-memory-utilization 0.96
  --reasoning-parser qwen3
  --enable-auto-tool-choice 
  --tool-call-parser qwen3_coder
  --safetensors-load-strategy lazy
  --enable-prefix-caching 
  --max-model-len auto
  --enable-chunked-prefill
  --speculative-config '{\"method\":\"mtp\",\"num_speculative_tokens\":3}' 

I don't have this error. I'm using VLLM_PLE_CPU_OFFLOAD=1 env var to offload the ngram to the cpu (cf https://recipes.vllm.ai/Qwen/Qwen3.8-Flash-Next )

check https://old.reddit.com/r/LocalLLM/comments/1vz20ap/qwen38flashnextnvfp4_on_single_rtx_pro_6000_120ts/

however I get an OOM when the weights are loaded into a single RTX 6000 Pro

I don't have this error. I'm using VLLM_PLE_CPU_OFFLOAD=1 env var to offload the ngram to the cpu (cf https://recipes.vllm.ai/Qwen/Qwen3.8-Flash-Next )

check https://old.reddit.com/r/LocalLLM/comments/1vz20ap/qwen38flashnextnvfp4_on_single_rtx_pro_6000_120ts/

however I get an OOM when the weights are loaded into a single RTX 6000 Pro

I have not tried this one - But I got the Inferact version working on a single RTX Pro 6000. It is very fast and runs well. Switching over to this model from DSv4-Flash for now.

Qwen3.8-Flash-Next NVFP4 on single RTX PRO 6000 (TP=1): fixed the weight_scale + pidfd_getfd crashes
Got RadixArk/Qwen3.8-Flash-Next-NVFP4 running on a single RTX PRO 6000 Blackwell (96GB, TP=1, MTP enabled, 262K ctx) after hitting three different startup crashes. Sharing the fixes:

  1. ValueError: There is no module or parameter named 'ngram_embedding.weight_scale'
    Root cause: the NVFP4 checkpoint stores the PLE shards as global-scale FP8 (ngram_embedding.shard_N.weight F8_E4M3 + ngram_embedding.weight_scale) while listing .ple. under ignore in the quant config. The stock _get_ple_embedding_quant_method() gate in ple_layer.py requires an Fp8Config and returns None for the ModelOpt/NVFP4 config — so the weight loader never gets the FP8 embedding method and dies on weight_scale. Fix: small patch that returns Qwen3_8FlashNextPLEFp8EmbeddingMethod() when VLLM_PLE_FP8_CHECKPOINT=1 is set (stock behaviour otherwise). Self-contained Dockerfile, no extra files:
FROM vllm/vllm-openai:qwen38-flash-next

RUN python3 - <<'EOF'
p = "/usr/local/lib/python3.12/dist-packages/vllm/models/qwen3_8_flash_next/nvidia/ple_layer.py"
s = open(p).read()

old_import = "import math\n"
new_import = "import os\nimport math\n"
assert s.count(old_import) == 1
s = s.replace(old_import, new_import, 1)

old_gate = (
    "    if not isinstance(quant_config, Fp8Config):\n"
    "        return None\n"
)
new_gate = (
    "    if not isinstance(quant_config, Fp8Config):\n"
    '        if os.environ.get("VLLM_PLE_FP8_CHECKPOINT", "0").lower() in ("1", "true"):\n'
    "            return Qwen3_8FlashNextPLEFp8EmbeddingMethod()\n"
    "        return None\n"
)
assert s.count(old_gate) == 1
s = s.replace(old_gate, new_gate, 1)

open(p, "w").write(s)
print("ple_layer.py patched")
EOF
  1. RuntimeError: pidfd_getfd: Operation not permitted
    Only happens with VLLM_PLE_CPU_OFFLOAD=1 (which you need on 96GB to keep the 51B ngram table out of VRAM). The PLE offload worker is a separate process that receives CUDA tensors from the worker via torch IPC, which uses pidfd_getfd to dup the fd across processes — that syscall requires CAP_SYS_PTRACE. Fix: --cap-add=SYS_PTRACE for docker, or in k8s:
    securityContext:
    capabilities:
    add:
    • SYS_PTRACE
  2. RuntimeError: Insufficient space in /dev/shm: 160 MiB required, 64 MiB free
    k8s default /dev/shm is 64MiB. Fix: --shm-size=1g for docker, or an emptyDir: {medium: Memory} volume mounted at /dev/shm in k8s.
    Working run command (docker):
    VLLM_PLE_CPU_OFFLOAD=1 VLLM_PLE_FP8_CHECKPOINT=1
    docker run --gpus all --shm-size=1g --cap-add=SYS_PTRACE
    -v /path/to/model:/model-data -p 8000:8000
    vllm-qwen38-flash-next-nvfp4
    --model /model-data --served-model-name Qwen3.8-Flash-Next
    --max-model-len 262144 --gpu-memory-utilization 0.96
    --max-num-seqs 2 --max-num-batched-tokens 8192
    --speculative-config '{"method":"mtp","num_speculative_tokens":3}'
    --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3
    Model loads at ~79GB, PLE table offloaded to host RAM, MTP drafting works. Runs, though I'd benchmark before comparing tg speeds.

Sign up or log in to comment