LingBot-VA — W4A8 SVDQuant
4-bit weight / 8-bit activation post-training quantization of LingBot-VA (RoboTwin post-trained checkpoint), produced with an SVDQuant-style pipeline: per-channel SmoothQuant, a rank-32 FP16 low-rank branch, and a group-wise INT4 residual.
This repository contains the real quantized payload — packed INT4 codes — not a "fake-quant" checkpoint that stores dequantized FP16 weights.
Contents
| file | description |
|---|---|
lingbot_va_w4a8_svdquant.pt |
packed INT4 weights + scales + low-rank branch (2.58 GB) |
lingbot_va_w4a8_svdquant_meta.json |
scheme, group sizes, calibration provenance |
What is quantized
The 300 transformer block Linear layers of the LingBot-VA video-action DiT, matched by
(^|\.)blocks\.\d+\.. Embeddings, heads, the VAE and the text encoder are left in their
original precision.
params quantized 4.91 B
packed INT4 2340 MB
aux (fp16) 305 MB scales + rank-32 branch + smoothing vector + bias
effective 4.52 bits / param
on disk 2.58 GB (vs 9.82 GB at bf16 — 3.8x smaller)
Tensor format
Per layer:
| tensor | dtype | shape | meaning |
|---|---|---|---|
qweight |
uint8 | [out, in/2] |
two INT4 codes per byte |
wscale |
fp16 | [out, in/64] |
per-output-channel group-64 scale |
lr_a, lr_b |
fp16 | [32, in], [out, 32] |
rank-32 low-rank branch |
inv_smooth |
fp16 | [in] |
inverse SmoothQuant factor |
bias |
fp16 | [out] |
optional |
INT4 codes are stored as unsigned nibbles offset by +8 (stored 0..15, true range -8..7);
the low nibble is the even column.
Dequantization
lo = (qweight & 0x0F).to(torch.int16) - 8 # even columns
hi = ((qweight >> 4) & 0x0F).to(torch.int16) - 8 # odd columns
codes[:, 0::2], codes[:, 1::2] = lo, hi
W = (codes.reshape(out, in_f // 64, 64) * wscale.unsqueeze(-1)).reshape(out, in_f)
W = W + lr_b @ lr_a # smoothed weight
y = linear(quant_a8(x * inv_smooth), W) + bias
W is the smoothed weight, so the input must be scaled by inv_smooth before the matmul —
the two are inverse halves of one transform and separating them changes the layer's function.
Activations are quantized at runtime to INT8, per-token, group-64.
Calibration
50 RoboTwin episodes — one randomly sampled per task across all 50 tasks, seed 42 — using all 10,919 frames for the per-input-channel activation statistics.
Verification
Payload integrity. All 300 layers present with advertised shapes/dtypes; INT4 codes within
[-8, 7]; pack/unpack round-trips losslessly; dequantized weights reproduce the smoothed FP
weights to a worst-case relative error of 4.85e-04 (fp16 rounding from applying the low-rank
branch to the output rather than folding it into W).
Closed-loop behaviour. Run in RoboTwin simulation with the quantized weights actually driving the policy:
| task | success |
|---|---|
handover_block |
10/10 |
lift_pot |
10/10 |
scan_object |
9/10 |
These are 10-episode functional checks confirming the checkpoint behaves correctly end-to-end — they are not a benchmark success rate, and should not be quoted as one.
Performance note
Dequantizing per forward and running an ordinary GEMM is slower than bf16 (measured 2.8–6.9× slower on these layer shapes), because rebuilding the weight costs more than the reduced memory traffic saves. Realising a speedup requires a fused kernel that never materializes the weight. Note that fused SVDQuant kernels such as Nunchaku implement W4A4, not W4A8, so they cannot serve this checkpoint directly.
License
Derivative of LingBot-VA and inherits its licensing terms. Consult the base model's license before use or redistribution.