mlboydaisuke/litert-gpu-moe-fusion-bug-evidence

moe output + fused elementwise op = invalid GPU shader: run logs and recipes

Logs and recipes for a report on the LiteRT GPU delegate: when the moe custom op's only consumer is an elementwise op (the residual ADD in Qwen3-MoE-style models), ML Drift fuses that op into the moe kernel and the generated shader does not compile (WebGPU: WGSL parse error var vec4 : vec4<f32><f32>(sum)_final;; Metal: float4 float4(sum)_final;).

All runs: 2026-11-03 (JST) on an Apple M4 Max (128 GB), macOS 27.0 (26A428). The nine runs of run_repro.sh ran at 12:13 JST and the two of run_repro2.sh at 12:38 JST; each run is one process with its own copy of the model file and stdin from /dev/null.

• <name>.err.txt = stderr, <name>.out.txt = stdout (the generated text), <name>.cmd_rc.txt = the exact command, then the exit code, the wall time and line counts of err.txt (validation = lines with Validation error, final = lines with _final, newLibraryWithSource = Metal compile errors), then the model file's sha256.
• Local paths are shortened to $K (the folder of these runs) and $L (the folder of the source build). Nothing else in the logs was changed. In the scripts, the comments were rewritten and the two folders are taken from the environment.

Runs

file prefix	binary	model	backend	what it shows
stock-webgpu-gelu	LiteRT-LM v0.19.0 release litert_lm_main (macOS arm64, from litert_lm_macos_arm64.tar.gz, sha256 21d8e3952bae…)	litert-torch litert_moe export of a random 2-layer Qwen3-MoE, GELU-tanh, fp32 weights, no activation type set (fp16 activations), 160,429,936 bytes, sha256 6200b11a9ee1…	WebGPU (--backend=gpu)	exit 0, 483 Validation error lines; the first is the WGSL parse error (err.txt lines 28–37: var vec4 : vec4<f16><f16>(sum)_final;), the rest are the pipelines and command buffers built from that shader module; all 236 decoded tokens are token id 0, ten newlines each in this tokenizer (this CLI does not stop at --max_output_tokens; the counts are in the BenchmarkInfo at the end of out.txt)
stock-cpu-gelu	same	same	CPU	exit 0, text (random weights, so random text)
A-webgpu-fp32	build A: LiteRT-LM dab2e708 built from source with LiteRT dcb657e8 + PR #11192 + PR #11188 and ML Drift 76492bb9 (litert_lm_advanced_main, sha256 a545652bcef5…, with the WebGPU accelerator library)	a random 3-layer SiLU Qwen3-MoE (16 experts, top-8, fp32 experts, fp32 activations declared) written with LiteRT's Tensor API, sha256 f29e019671e7…	WebGPU	exit 0, 75 Validation error lines, err.txt lines 111–125 the WGSL parse error (var vec4 : vec4<f32><f32>(sum)_final;), output ! × 32 (token id 0)
A-webgpu-fp32_g	same	the same graph with the workaround x = (x + moe) + moe * 0, sha256 23fbd18d23e2…	WebGPU	exit 0, 0 validation errors, output byte-identical to A-cpu-fp32.out.txt
A-metal-fp32	build A with the Metal accelerator library	the plain graph	Metal	exit 2, err.txt lines 72–96: newLibraryWithSource: program_source:57:22: error: expected ';' at end of declaration float4 float4(sum)_final;
A-metal-fp32_g	same	the workaround graph	Metal	exit 0, output byte-identical to the CPU output
A-cpu-fp32	build A, no accelerator library	the plain graph	CPU	exit 0, the reference text
A-cpu-fp32_g	same	the workaround graph	CPU	exit 0, the same text as A-cpu-fp32
A-webgpu-gelu	build A, WebGPU	the litert-torch GELU-tanh export above	WebGPU	exit 0, 75 validation errors, the vec4<f16> form (err.txt lines 108–116), output token id 0 × 32 (ten newlines each)
A-metal-gelu	build A, Metal	the litert-torch GELU-tanh export (fp16 activations)	Metal	exit 2, err.txt lines 71–72: newLibraryWithSource: program_source:57:10: error: cannot initialize a variable of type 'half4' … with an lvalue of type 'float4' half4 half4(sum)_final;
A-webgpu-fp32-shif	build A, WebGPU	the plain 3-layer graph	WebGPU, --sampler_handles_input=false	exit 0, 73 validation errors, the same WGSL parse error, output ! × 32

Graph and recipes

• graph_around_moe_decode_prefill32.txt — the ops around the moe op in the plain graph (prefill-32 signature): … TOPK_V2 → SUM → DIV → RESHAPE → CUSTOM:moe (7 inputs, custom options {activation silu, hidden_dim 64, model_dim 128, num_active_experts 8, num_experts 16, renormalized_top_weights true, weight_type fp32}) → ADD (residual) → MUL/MEAN/… (RMSNorm). The moe output's only consumer is the ADD.
• graph_around_moe_guarded.txt — the same place in the workaround graph: the moe output feeds the ADD and a MUL by 0, whose result is added afterwards.
• tiny_gelu_litert_torch_config.json — the HF config of the GELU-tanh model (Qwen3MoeForCausalLM, hidden 128, 2 layers, 8 experts, top-2, hidden_act gelu_pytorch_tanh, vocab 151936), exported with litert-torch's litert_moe MoE implementation (prefill 32, KV cache 256) and packed with litert-lm-builder.
• tiny_qwen3moe_config.json, tiny_qwen3moe_tiny.json — the HF config and the seed / sha256 of the 3-layer SiLU model (hidden 128, 3 layers, 16 experts, top-8, norm_topk_prob true, q/k norm, seed 20261102).
• build_tiny.sh — how the SiLU model files were made: random weights → packed weights → a .tflite written by a C++ writer built on LiteRT's Tensor API, with and without --moe_fusion_guard → .litertlm with fp32 activations declared. The tools it calls are not attached.
• run_repro.sh — the nine runs; run_repro2.sh — the two later runs (A-webgpu-fp32-shif, A-metal-gelu).
• SHA256SUMS.bundles — sha256 of the .tflite and .litertlm files of the SiLU model.

The model files (160 MB each, most of it the embedding table of the full 151,936-token vocabulary) are not attached; they can be shared.