Duplicated from  local-inference-lab/Qwen3.8-Flash-Next-NVFP4

Mia-AiLab

/

Qwen3.8-Flash-Next-NVFP4

Like 12

Summary

I evaluated Mia-AiLab/Qwen3.8-Flash-Next-NVFP4 as a coding-support model for Codex on one NVIDIA GB10 DGX Spark. This is a third-party serving report: it does not modify or redistribute the checkpoint, and it is not an endorsement by Mia or the Qwen authors.

The Mia repository currently identifies this checkpoint as a mirror of local-inference-lab/Qwen3.8-Flash-Next-NVFP4. The runtime recipe used here is based on the public Mia single-DGX deployment kit.

Validated host profile

• vLLM, tensor parallelism 1, native context 262144.
• Maximum two simultaneous sequences; chunked prefill 2048.
• FP8 KV cache, gpu_memory_utilization=0.747, 30 GiB host reserve and 12 GiB KV target.
• MTP with 3 speculative tokens; dynamic MTP_K_SCHEDULE disabled.
• FULL_DECODE_ONLY CUDA graphs with capture sizes [4, 8]; compilation mode 0.
• Checkpoint-default recurrent state (float32). BF16 recurrent state remains an unpromoted experiment for this coding workload.
• VLLM_USE_V2_MODEL_RUNNER=1 is not set. In the matched comparison this build already used the internal V2 worker path, so the explicit flag produced no reproducible operational gain.

The profile is deliberately conservative for a shared unified-memory host. It should be retuned on other DGX Spark systems.

Request-level tuning for a Codex support coder

The wrapper performs local task pre-analysis and selects one of four bounded profiles:

low and preserve_thinking remain explicit experimental or diagnostic paths. Generated code is accepted only after tests and independent review by the orchestrator.

Coding-support measurements

The private frozen harness contained 24 coding cases, 2 genuine missing-context controls and 212 initial tests. Raw prompts, source code and generated patches are withheld because they come from private projects; the aggregates and exact conditions are reported here.

• Flash NVFP4/vLLM passed 19/24 coding cases on the first attempt and 20/24 after one feedback retry in the local matched series. This is not a SWE-bench or LiveCodeBench score.
• In a small paired ablation (n=3, low profile), client medians were 17.171 s for MTP3, 37.618 s for MTP1 and 49.601 s for MTP0. This supports retaining MTP3 on this host, not a universal optimum.
• Reusing an authorized identical ~31.8k-token prefix reduced a three-request pilot median from 20.528 s to 7.438 s (63.76%). The gain requires an actually reusable prefix and does not reduce decode cost.
• Adding the explicit V2 runner flag changed client median from 15.788 s to 15.750 s and first-attempt QA from 21/24 to 20/24. I therefore do not advertise it as an optimization for this build.
• The orchestration layer passed 138 tests and 3 subcases across five runs. A final repeated gate produced 17/18 valid low-profile terminal results and 8/9 valid results on the more complex Flash set.

Interpretation and limits

The main contribution is a reproducible serving and evaluation recipe for Codex-style delegation, including task-aware routing, explicit sampling, test-gated acceptance and honest negative results. Several runtime knobs (MTP3, FP8 KV and chunked prefill) are already present in the Mia deployment kit; these measurements add a conservative shared-host profile and coding-workload evidence rather than a new model quantization.

The following are intentionally not promoted as defaults: dynamic MTP scheduling, BF16 recurrent state for this workload, automatic low, automatic preserved thinking, and the explicit V2 environment flag. The third-party gated ABLIT checkpoint was excluded from this stock-model report.

If useful, I can provide the aggregate JSON profile and benchmark manifest as a small community documentation update, while keeping weights, credentials, private prompts and private source code out of the model repository.