dexgarmentlab-dual-dynamics-gps

GPSDynamicsModel — graph-based (GNN + Transformer) dynamics model for variable-vertex cloth meshes, two-gripper. Given the 3 previous mesh frames and a per-hand 3D gripper action, predicts the next 5 mesh frames via DDPM diffusion, using each cloth's own rest state and topology (no global template).

• Task data: DexGarmentLab bimanual garment demos (dexgarmentlab_dual_gps2048_clean.h5, Cloth-splatters/dexgarmentlab-dual-meshes-clean)
• Formulation: DDPM diffusion (scaled_linear, 1000 steps, Min-SNR-gamma 5.0)
• Max grippers: 2
• Cross-attention mode: parallel
• Max vertices per mesh: 2048
• Best validation loss: 6.656742061750265e-05 m^2 position MSE (checkpoint in model/ is checkpoint-best)
• Training run: demos2h_dyn_gps_ddpm_2026-08-21_17-27-35_776086 (full config in config.yml)

Recipe

lr 5e-4 cosine over an explicit 160k-step horizon, warmup 500, batch 32, history_noise_std 1e-4. The cosine horizon is not cosmetic: a constant lr 1e-3 run on the predecessor dataset reached 4.89e-4 around step 155k and then diverged to ~1e-2 by 200k. Best validation landed at step 155k of 160k with the LR already annealed to ~0, so the horizon — not the model — was the binding constraint; a longer run would likely still gain.

Structured history corruption (history_rigid_std / history_smooth_std) is deliberately not used: A/B'd against a matched control it lost on both ground-truth-bootstrapped (3.48 vs 2.94 cm) and state-estimator-fed (4.91 vs 4.20 cm) histories.

Two-hand action convention

max_grippers: 2. The action is one token stream per hand, with a learned gripper-ID embedding and per-gripper grasped-flag node channels. A trajectory that moves only one hand parks the other and flags it in the recording's hand_active attribute; the loader masks that slot out of both gripper_mask and the translation-centering centroid, so it behaves exactly like a padded slot.

This matters when reading the source data: the recording also carries legacy single-gripper gripper_pos / actuated_vertices fields, but on a two-handed trajectory those are a lossy back-compat projection — the legacy point is a virtual gripper between the hands, ~14 cm from the vertices it nominally drives (the real per-hand positions sit 3-4 cm away), and the legacy mask covers only ~85 of the ~153 vertices the two hands actually hold. Use gripper_pos_2h / actuated_vertices_2h, as ClothDynamicsVariableDataset and ClothDynamicsGraphDataset now do.

Evaluation

scripts/benchmark_dynamics_accuracy.py, validation split (24 held-out garments — the split is by garment, baked into the dataset file, so no trajectory of a validation cloth is ever seen in training), 50 denoising steps, one trajectory per cloth for the rollout:

model	one-shot (5 frames)	rollout mean	rollout end
-dynamics-gps (DDPM)	1.25 cm	1.86 cm	3.00 cm
-dynamics-gps-flow	1.24 cm	1.84 cm	3.10 cm
-dynamics-gns	0.52 cm	3.75 cm	5.72 cm

The two diffusion models are within noise of each other on both metrics. GNS is 2.4x better one step ahead and ~2x worse over the horizon — it is the accurate one-step predictor and the fastest to drift.