Repository: simpler-env/SimplerEnv
Issue: How to adjust camera angle to be a simulated camera on top of the robotic arm?
Root cause: Documentation Deficiency

Hello,

How can I adjust the camera angle of the simulation for the robot arm such that the camera is attached to the robot (that is: I want my camera angle to be an angle that is a birds-eye view from the robotic arm itself and the camera view moves with the robotic arm as the arm moves)?

Thanks!

You can see how the google robot's camera is mounted on the robot's camera link and see the camera pose wrt robot link. You can set a fixed transformation between a robot link and the camera.

Ok, thanks for the comment! I'm also experimenting with using Simpler for OpenVLA experiments and was looking through the openvla policy pull request.

How can I run the OpenVLA model on Simpler to reproduce the Bridge dataset results on the pull request? It seems different from running the example.ipynb script in the main repository.

I previously tried running the instructions that another user had but got these errors:
```
~/simpler-env/SimplerEnv$ python tools/calc_metrics_evaluation_videos.py --task=bridge_put_on
***Bridge Put On Env results***
********** Results for put_spoon_on_tablecloth **********
/home/alanw2/miniconda3/envs/simpler/lib/python3.10/site-packages/numpy/core/fromnumeric.py:3464: RuntimeWarning: Mean of empty slice.
  return _methods._mean(a, axis=axis, dtype=dtype,
/home/alanw2/miniconda3/envs/simpler/lib/python3.10/site-packages/numpy/core/_methods.py:192: RuntimeWarning: invalid value encountered in scalar divide
  ret = ret.dtype.type(ret / rcount)
WARNING: avg_sim_success is nan for ./results//rt_1_x_tf_trained_for_002272480_step/bridge_table_1_v1/arm_pd_ee_target_delta_pose_align2_gripper_pd_joint_pos/PutSpoonOnTableClothInScene-v0
WARNING: avg_sim_success is nan for ./results//octo-base/bridge_table_1_v1/arm_pd_ee_target_delta_pose_align2_gripper_pd_joint_pos/PutSpoonOnTableClothInScene-v0_octo_init_rng_0
WARNING: avg_sim_success is nan for ./results//octo-base/bridge_table_1_v1/arm_pd_ee_target_delta_pose_align2_gripper_pd_joint_pos/PutSpoonOnTableClothInScene-v0_octo_init_rng_2
WARNING: avg_sim_success is nan for ./results//octo-base/bridge_table_1_v1/arm_pd_ee_target_delta_pose_align2_gripper_pd_joint_pos/PutSpoonOnTableClothInScene-v0_octo_init_rng_4
WARNING: avg_sim_success is nan for ./results//octo-small/bridge_table_1_v1/arm_pd_ee_target_delta_pose_align2_gripper_pd_joint_pos/PutSpoonOnTableClothInScene-v0_octo_init_rng_0
WARNING: avg_sim_success is nan for ./results//octo-small/bridge_table_1_v1/arm_pd_ee_target_delta_pose_align2_gripper_pd_joint_pos/PutSpoonOnTableClothInScene-v0_octo_init_rng_2
WARNING: avg_sim_success is nan for ./results//octo-small/bridge_table_1_v1/arm_pd_ee_target_delta_pose_align2_gripper_pd_joint_pos/PutSpoonOnTableClothInScene-v0_octo_init_rng_4
WARNING: avg_sim_success is nan for ./results//openvla-7b/bridge_table_1_v1/arm_pd_ee_target_delta_pose_align2_gripper_pd_joint_pos/PutSpoonOnTableClothInScene-v0
sim visual matching partial success {'rt-1-x': nan, 'octo-base': nan, 'octo-small': nan, 'openvla-7b': nan}
real partial success {'rt-1-x': 0.042, 'octo-base': 0.5, 'octo-small': 0.542, 'openvla-7b': 0.1}
visual matching MMRV (partial success) 0.254
visual matching pearson correlation (partial success)  nan
avg kruskal (partial success):
       each checkpoint kruskal:
             KruskalResult(statistic=0.9999999999997733, pvalue=0.3173105078629661)
             KruskalResult(statistic=15.66666666666664, pvalue=7.554389326826452e-05)
             KruskalResult(statistic=17.45714285714282, pvalue=2.938580582975732e-05)
             KruskalResult(statistic=2.0434782608695747, pvalue=0.15285977016579425)
sim visual matching success {'rt-1-x': nan, 'octo-base': nan, 'octo-small': nan, 'openvla-7b': nan}
real success {'rt-1-x': 0.0, 'octo-base': 0.333, 'octo-small': 0.417, 'openvla-7b': 0.1}
visual matching MMRV 0.2125
visual matching pearson correlation nan
avg kruskal:
       each checkpoint kruskal:
             all same, 1.0
             KruskalResult(statistic=9.399999999999977, pvalue=0.002169854427130147)
             KruskalResult(statistic=12.368421052631579, pvalue=0.0004366567257915252)
             KruskalResult(statistic=2.0434782608695747, pvalue=0.15285977016579425)
********************



********** Results for put_carrot_on_plate **********
WARNING: avg_sim_success is nan for ./results//rt_1_x_tf_trained_for_002272480_step/bridge_table_1_v1/arm_pd_ee_target_delta_pose_align2_gripper_pd_joint_pos/PutCarrotOnPlateInScene-v0
WARNING: avg_sim_success is nan for ./results//octo-base/bridge_table_1_v1/arm_pd_ee_target_delta_pose_align2_gripper_pd_joint_pos/PutCarrotOnPlateInScene-v0_octo_init_rng_0
WARNING: avg_sim_success is nan for ./results//octo-base/bridge_table_1_v1/arm_pd_ee_target_delta_pose_align2_gripper_pd_joint_pos/PutCarrotOnPlateInScene-v0_octo_init_rng_2
WARNING: avg_sim_success is nan for ./results//octo-base/bridge_table_1_v1/arm_pd_ee_target_delta_pose_align2_gripper_pd_joint_pos/PutCarrotOnPlateInScene-v0_octo_init_rng_4
WARNING: avg_sim_success is nan for ./results//octo-small/bridge_table_1_v1/arm_pd_ee_target_delta_pose_align2_gripper_pd_joint_pos/PutCarrotOnPlateInScene-v0_octo_init_rng_0
WARNING: avg_sim_success is nan for ./results//octo-small/bridge_table_1_v1/arm_pd_ee_target_delta_pose_align2_gripper_pd_joint_pos/PutCarrotOnPlateInScene-v0_octo_init_rng_2
WARNING: avg_sim_success is nan for ./results//octo-small/bridge_table_1_v1/arm_pd_ee_target_delta_pose_align2_gripper_pd_joint_pos/PutCarrotOnPlateInScene-v0_octo_init_rng_4
WARNING: avg_sim_success is nan for ./results//openvla-7b/bridge_table_1_v1/arm_pd_ee_target_delta_pose_align2_gripper_pd_joint_pos/PutCarrotOnPlateInScene-v0
sim visual matching partial success {'rt-1-x': nan, 'octo-base': nan, 'octo-small': nan, 'openvla-7b': nan}
real partial success {'rt-1-x': 0.167, 'octo-base': 0.5, 'octo-small': 0.208, 'openvla-7b': 0.1}
visual matching MMRV (partial success) 0.14375
visual matching pearson correlation (partial success)  nan
avg kruskal (partial success):
       each checkpoint kruskal:
             KruskalResult(statistic=4.272727272727169, pvalue=0.0387287232951513)
             KruskalResult(statistic=15.66666666666664, pvalue=7.554389326826452e-05)
             KruskalResult(statistic=5.4651162790697425, pvalue=0.019399763184203167)
             KruskalResult(statistic=2.0434782608695747, pvalue=0.15285977016579425)
sim visual matching success {'rt-1-x': nan, 'octo-base': nan, 'octo-small': nan, 'openvla-7b': nan}
real success {'rt-1-x': 0.0, 'octo-base': 0.25, 'octo-small': 0.083}
Traceback (most recent call last):
  File "/home/alanw2/simpler-env/SimplerEnv/tools/calc_metrics_evaluation_videos.py", line 1058, in <module>
    calc_bridge_put_on_env_stats(args.log_dir_root)
  File "/home/alanw2/simpler-env/SimplerEnv/tools/calc_metrics_evaluation_videos.py", line 1012, in calc_bridge_put_on_env_stats
    mean_maximum_rank_violation(list(sim_visual_matching_success.values()), list(real_success.values())),
  File "/home/alanw2/simpler-env/SimplerEnv/simpler_env/utils/metrics.py", line 234, in mean_maximum_rank_violation
    assert perf_sim.shape == perf_real.shape
AssertionError```

This script you have run is for calculating metrics according to our paper, not for OpenVLA evaluation.

To evaluate OpenVLA, run the scripts under `scripts/` in https://github.com/DelinQu/SimplerEnv-OpenVLA/tree/main