FIRe-PegInsert-rewonly

FIRe residual policy (rl_games PPO) trained in Isaac Lab on the Forge PegInsert task, on top of the frozen GR00T VLA bhe1004/FIRe-VLA-GR00T-PegInsert-Sim. The final command is a = a_VLA + a_res (residual scale 1.0).

Ablation: force in reward only. The contact force (ft_force, force_threshold) is removed from the residual policy's observation (actor) and from the critic's privileged state. The force-aware reward is kept.

Setting

Item	Value
Task	Forge peg_insert, episode 10 s
Actor observation	fingertip_pos_rel_fixed, fingertip_quat, ee_linvel, ee_angvel + previous action (no force)
Critic state	fingertip_pos, fingertip_quat, ee_linvel, ee_angvel, joint_pos, held_pos, held_pos_rel_fixed, held_quat, fixed_pos, fixed_quat, task_prop_gains, ema_factor, pos_threshold, rot_threshold + previous action (no force)
Reward	r = r_s + r_p + r_sh (sparse success + force penalty + potential-based force shaping)
Success reward r_s	1 per step in success
Force penalty scale α	1e-05
Force shaping scale λ_f	1e-5 (effective; params/env.yaml lists force_shaping_lambda: 0.01, but the training code overrode it with 1e-5)
Reference force range	[5.0, 10.0] N (randomized)
Parallel envs / epochs / horizon	64 / 200 / 128
Network	LSTM 1024 x2 → MLP [512, 128, 64]
Seed	0

Result (simulation)

Metric (Episode/episode_success_env_ratio, all 64 envs)	Value
Mean of last 10 logged values	0.745
Max	0.844

Training run: wandb FIRe-PegInsert-rewonly-r2, 2026-10-18, item server (RTX A6000).

Files

rl_games log directory as produced by training:

• nn/ — checkpoints: Forge.pth, last_Forge_ep_100_rew_27.970821.pth, last_Forge_ep_200_rew_76.60746.pth, last_Forge_ep_200_rew__76.60746_.pth (Forge.pth = best during training)
• params/ — env.yaml, agent.yaml
• summaries/ — TensorBoard event file