lerobot

/

smolvla_robocasa

like 1

Follow

LeRobot 4.76k

Hi LeRobot team,

I am evaluating lerobot/smolvla_robocasa on RoboCasa with the documented command:

lerobot-eval \
  --policy.path=lerobot/smolvla_robocasa \
  --env.type=robocasa \
  --env.task=CloseFridge \
  --eval.batch_size=1 \
  --eval.n_episodes=2 \
  --eval.use_async_envs=false \
  --policy.device=cuda \
  '--rename_map={"observation.images.robot0_agentview_left": "observation.images.camera1", "observation.images.robot0_eye_in_hand": "observation.images.camera2", "observation.images.robot0_agentview_right": "observation.images.camera3"}'

The RoboCasa365 documentation says the policy input observation.state is a 16-dimensional proprioceptive state:

• base position
• base quaternion
• relative end-effector position
• relative end-effector quaternion
• gripper qpos

However, the loaded checkpoint config shows:

policy.input_features["observation.state"].shape = (6,)
policy.output_features["action"].shape = (12,)

while the environment config shows:

env.features["agent_pos"].shape = (16,)
env.features_map["agent_pos"] = "observation.state"

So during evaluation, the RoboCasa environment appears to provide a 16D state, but the released smolvla_robocasa checkpoint declares a 6D state input.

Could you clarify which one is expected?

1. Was smolvla_robocasa actually trained with the full 16D RoboCasa state?
2. Or was it trained with a 6D subset of the state?
3. If it was trained with 16D, should the checkpoint config.json / preprocessor metadata be updated from 6D to 16D?
4. If it was trained with 6D, should the RoboCasa365 documentation/model card mention the exact 6D state representation and how the 16D environment state is reduced?

This matters because users trying to evaluate or adapt the checkpoint need to know whether to feed the full RoboCasa 16D state, crop it to 6D, or retrain/fine-tune with a consistent 16D representation.

Thanks!