mesiu650/GR00T-N1.6-LIBERO-llm-w4a4-flatquant-tf32

GR00T N1.6 LIBERO — LLM W4A4 FlatQuant+GPTQ (TF32 transform)

ONNX for the language-model backbone of GR00T N1.6 (LIBERO checkpoint), quantized to W4A4 with a learned FlatQuant transform, LWC/LAC clipping and GPTQ int4 weights.

Build the TensorRT engine on the target device — engines are not portable across sm_89/TRT 10.15 and sm_87/TRT 10.3.

Files

file	size	md5
llm.onnx	406.8 MB	ec98712137dbad085bc48937370cd535
llm.onnx.data	790 KB	096dd86c9a4be1a090cda30e6db196ea

Configuration

transform	2-way Kronecker, (64, 96) at K=6144 and (32, 64) at K=2048
prologue	-DFQ_USE_TF32_TC=1 — transform on TF32 tensor cores
weights	FlatQuant + LWC/LAC + GPTQ, int4, per-output-row
activations	dynamic per-row int4
sites	all 4 GEMM sites (qkv, o_proj, gate+up, down_proj)

Validation

Closed-loop LIBERO, 800 episodes (4 suites x 10 tasks x 20 episodes, 520 max steps, ring offset 0, 4 denoising steps), DiT held at omega_qvla_w4a4_sq:

suite	SR
libero_spatial	0.990
libero_object	0.985
libero_goal	0.955
libero_10	0.900
overall	0.9575

Reference with the same config but an fp32 transform: 0.9613 (800 episodes). TF32 costs ~3 episodes in 800 and is worth 13.86 → 10.89 ms on Jetson AGX Orin (projected; the Orin re-timing of this ONNX is still outstanding).

Offline stage-H metrics: chan_rel_err median 0.2487, tok_cos_ex_top4_min +0.0292. That positive sign is the property distinguishing FlatQuant+GPTQ from earlier W4A4 configurations.

TensorRT build check: BF16-vs-TRT cosine 0.9916.

Note on the factorization

A faster factorization — (128,48)/(64,32), 16% quicker on the Orin's TF32 prologue — fails the stage-H gate: tok_cos_ex_top4_min drops to −0.1009, below the fixed-Hadamard baseline, despite carrying 40% more transform parameters. It is not published here. See FLATQUANT_W4A4_LLM.md §18 in the source repo.

Latency

Latency must be measured on the deployment target. The same ONNX has measured 4.97 ms (FlatQuant) vs 3.09 ms (INT8) on an RTX 4090 and 19.38 vs 10.22 ms on a Jetson AGX Orin — opposite conclusions, because sm_87 and sm_89 differ in INT4 tensor-core throughput.