mewbot97/Intent2Action-glimmer-push

Intent2Action · Glimmer with push experience (G1e)

A QLoRA adapter for Muse Glimmer 30B: the recovery agent G1d's training data plus experience of objects the gripper cannot hold. When pick fails because an object is wider than the gripper opens, it says why and asks the operator whether to push it; after a yes, it pushes the object onto its goal with the robot's force-limited moves (move_to_contact, move_compliant), checks where it is, and patches the rest of the program down to a check. Every robot move it issues still needs a person's approval in Intent2Action.

Training

• Data: G1d's 735 rows plus 393 push-experience rows: 200 asks, 161 pushes, 32 declines, generated on the plan tier (learn/push_experience.py).
• Objects: a crate, a tray and a tote, onto a mat or a pad. The carton and the painted zone of the evaluation are held out.
• Method: QLoRA with the cookbook recipe (r 32, alpha 16), 10,240-token sequences (rows with tool calls).

Results

	G1d	G1e
Held-out carton: asks, saying why	0/10	15/15
Held-out carton: pushed onto the zone (within 2 cm)	0/10	11/15
250 held-out program tasks × 3 (plan tier)	83%	77%
of which stacking	89%	58%
Isaac physics, 50 tasks	42/50	40/50
Live recovery, 20 tasks	9/20	10/20

The push experience costs stacking: the model writes "outermost first" for towers. A re-balanced mix is the next step. Details: docs/designs/learning_results.md in the repository.

License

Apache-2.0, as the base model. This adapter is a derivative of Muse Glimmer 30B.