Qwen3Next
#1
by coe-arrtificial - opened
I tried to run this model with vllm and lots of flags, but still getting "RuntimeError: No Qwen3Next layer found in the model.layers."
Any idea why ?
My flags
--speculative-config '{"method": "mtp", "num_speculative_tokens": 3, "disable_padded_drafter_batch": true}'
--language-model-only --skip-mm-profiling
--enable-prefix-caching
--enable-cumem-allocator
--no-disable-cascade-attn
--max-num-batched-tokens 8196
--max-num-seqs 6
--enable-flashinfer-autotune
--enable-chunked-prefill
--kv-cache-dtype fp8
--load-format fastsafetensors
--reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder