dexgarmentlab-dual-dynamics-gps-flow

GPSDynamicsModel — graph-based (GNN + Transformer) dynamics model for variable-vertex cloth meshes, two-gripper, trained with rectified flow matching instead of DDPM. Given the 3 previous mesh frames and a per-hand 3D gripper action, predicts the next 5 mesh frames.

• Task data: DexGarmentLab bimanual garment demos (dexgarmentlab_dual_gps2048_clean.h5, Cloth-splatters/dexgarmentlab-dual-meshes-clean)
• Formulation: flow matching (FlowMatch_DynamicsGPS, logit-normal t, shift 1.0)
• Max grippers: 2
• Cross-attention mode: sequential
• Max vertices per mesh: 2048
• Best validation loss: 7.857939211438254e-05 m^2 position MSE at 25 inference steps (checkpoint in model/ is checkpoint-best)
• Training run: demos2h_dyn_gps_flow_2026-08-21_17-27-35_776087 (full config in config.yml)

Recipe

lr 2e-4 cosine over a 200k-step horizon, warmup 2000, batch 32, history_noise_std 1e-4. Validation plateaued at step 175k — the last 25k steps gained nothing, so this horizon was long enough (unlike the DDPM arm).

cross_attn_mode: sequential here, versus parallel for the DDPM sibling. That is carried over from the flow recipe it inherits and is the setting this checkpoint's numbers belong to; parallel vs sequential has never been A/B'd within the dynamics family, so treat the choice as inherited rather than validated.

Two-hand action convention

max_grippers: 2. The action is one token stream per hand, with a learned gripper-ID embedding and per-gripper grasped-flag node channels. A trajectory that moves only one hand parks the other and flags it in the recording's hand_active attribute; the loader masks that slot out of both gripper_mask and the translation-centering centroid, so it behaves exactly like a padded slot.

This matters when reading the source data: the recording also carries legacy single-gripper gripper_pos / actuated_vertices fields, but on a two-handed trajectory those are a lossy back-compat projection — the legacy point is a virtual gripper between the hands, ~14 cm from the vertices it nominally drives (the real per-hand positions sit 3-4 cm away), and the legacy mask covers only ~85 of the ~153 vertices the two hands actually hold. Use gripper_pos_2h / actuated_vertices_2h, as ClothDynamicsVariableDataset and ClothDynamicsGraphDataset now do.

Evaluation

scripts/benchmark_dynamics_accuracy.py, validation split (24 held-out garments — the split is by garment, baked into the dataset file, so no trajectory of a validation cloth is ever seen in training), 50 denoising steps, one trajectory per cloth for the rollout:

model	one-shot (5 frames)	rollout mean	rollout end
-dynamics-gps (DDPM)	1.25 cm	1.86 cm	3.00 cm
-dynamics-gps-flow	1.24 cm	1.84 cm	3.10 cm
-dynamics-gns	0.52 cm	3.75 cm	5.72 cm

The two diffusion models are within noise of each other on both metrics. GNS is 2.4x better one step ahead and ~2x worse over the horizon — it is the accurate one-step predictor and the fastest to drift.