armanakbari4/fastwam-ur3-svdquant-nvfp4

FastWAM UR3 3-task — SVDQuant W4A4 in NVFP4

SVDQuant (Li et al., ICLR 2025) applied to the UR3 3-task Fast-WAM fine-tune (step 7000), in NVFP4 rather than INT4, for Blackwell GPUs. Companion to the INT4 export at arashakb/FastWAM_UR3; this is the same method on the number format the RTX 5090 executes natively.

	this checkpoint	ur3_step7000_svdquant_w4a4.pt (INT4)
numeric format	NVFP4 — E2M1, block 16, E4M3 block scales	INT4, group 64, fp16 scales
weights / activations	W4A4, both block 16	W4A4, both group 64
smoothing alpha	per-layer grid search	fixed 0.5
BPW (weights + low-rank)	4.8353	4.5798
kernels	Nunchaku SVDQW4A4Linear, precision nvfp4	QuantWAM Triton, packed INT4 on INT8 MMA

NVFP4 costs more bits than the INT4 export, not fewer: one fp8 scale per 16 weights plus a bf16 per-output-channel scale, against one fp16 scale per 64. That is the price of the finer granularity and it is why the BPW column reads 4.84 rather than 4.58.

The same 600 Linears are quantised as in the INT4 export — both experts' 30 blocks x {self_attn q/k/v/o, cross_attn q/k/v/o, ffn.0, ffn.2}, 5.914 B parameters. Embeddings, heads, the action encoder, norms, modulation and the proprio encoder stay bf16. The file is self-contained: quantised Linears plus every unquantised mot tensor plus the proprio encoder, so the 11.2 GiB bf16 checkpoint is not needed at inference.

What is different from the INT4 export, and why

The smoothing strength is searched per layer. SVDQuant's paper selects alpha by grid search on the calibration set; the INT4 export used a fixed 0.5 scored on weight reconstruction error. Here alpha is chosen per layer from {0.25, 0.375, 0.5, 0.625, 0.75}, scored on that layer's own output error against real calibration activations, which is what the paper specifies.

Only 193 of 600 layers chose 0.5:

alpha: 0.25 -> 2,  0.375 -> 19,  0.5 -> 193,  0.625 -> 199,  0.75 -> 187

The low-rank branch and residual are recomputed, not converted. SVDQuant's refinement loop is qW = Q_w(W_hat - L), so the optimal rank-32 branch depends on which quantiser Q_w is. Reusing the INT4 solution and re-encoding its codes would be double quantisation and would not be an NVFP4 SVDQuant. Everything is rebuilt from the bf16 weights.

Packing is DeepCompressor's, via convert_to_nunchaku_w4x4y16_linear_weight with float_point=True — their code, not a reimplementation, so the E2M1 codebook and the fp8 scale swizzle are the reference ones.

Calibration provenance

Two different things, and the checkpoint's meta records them separately because they have different pedigrees:

• Activation absmax is recovered analytically from the INT4 export rather than re-measured: a = s^2 * w_absmax, the inverse of compute_smooth at alpha = 0.5. These are the exact values the original 9908-observation cluster calibration produced, so alpha is the only thing that differs between this export and the INT4 one. 37 of 1,781,760 channels sit on compute_smooth's 1e-5 input clamp and come back as 1e-5.
• The alpha search was scored on 1663 observations — all 30 calibration episodes (10/task x 3 tasks, seed 42), frames strided 1-in-6, reservoir-sampled to 256 rows per Linear. That is a subsample of the 9908, and it is a deliberate deviation recorded rather than implied.

The activation capture used PyAV instead of torchcodec (the build box has no FFmpeg 7, which torchcodec requires, and FFmpeg 4 cannot decode the AV1 videos at all). Both decode through libdav1d; the residual risk is a swscale +/-1 LSB difference on some pixels.

Verification, and its limits

Open-loop agreement with bf16 on real captured robot frames, horizon 64, N = 12:

	NRMSE
executed window (steps 0:16, what the client commands)	0.0008
full chunk (0:64)	0.0027
correlation	0.999952

Against recorded teleoperation on held-out episodes (N = 54, 3 episodes x 6 frames x 3 tasks, horizon 32, executed window):

arm	vs recorded
bf16	0.00781
this checkpoint	0.00771
deflated-ASP NVFP4	0.00782

Read that table carefully. All three land within 1e-4 of each other on a base of 7.8e-3, and both quantised arms come out at or below bf16, which cannot be a real effect. The model's own error against a human demonstrator is roughly 6x the quantisation error, so this metric is saturated and does not distinguish W4A4 arms. It is reported because it is what was measured, not because it separates anything.

No closed-loop success rate has been measured with this checkpoint. Open-loop agreement is necessary and not sufficient.

Running it

Needs Nunchaku with an NVFP4 build and a Blackwell GPU. On an RTX 5090, measured end to end (full infer_action, CUDA-graph replayed, bf16 measured in the same process):

	weights	latency
bf16	12.55 GiB	131 ms
this checkpoint	4.94 GiB	178 ms

It is 1.36x slower than bf16 on this card, and that is not a defect of SVDQuant. At the action expert's 64 tokens the GEMMs are far below the size where 4-bit weight traffic pays: Nunchaku pads the batch to 256, so an M=64 call computes four times the rows asked for, and the kernel time is flat from M=16 to M=256. The crossover against bf16 is around M=512, and by M=1024 the same kernels reach 1.4x. Quantisation buys 2.5x memory here, not speed.

from fastwam_nvfp4_runtime import install_nvfp4     # in QuantWAM/adapters/fastwam/
install_nvfp4(model, "ur3_step7000_svdquant_nvfp4.pt")

install_nvfp4 raises unless every name in the checkpoint resolves to an nn.Linear: a partial swap still runs and still emits actions, and the numbers it produces belong to no scheme at all.

You also need the Wan2.2 VAE to encode the camera image, the dataset stats for proprio z-scoring and action denormalisation, and the pre-encoded prompt embeddings — all as for the INT4 export.

Tasks

• blue_basket — put the medicine then the measuring tape inside the blue basket
• drawer — open the drawer, put the white box inside the drawer then close the drawer
• stacking_cubes — put the green cube on top of the black cube and put the red cube on top of the green cube