zhiminy/EvalEng huggingface dataset:
Repository: stepjam/RLBench
Issue: Example RL algorithms or other repos utilizing RLBench?

I'm looking into using RLBench, thanks for creating it! Are any baseline algorithms already integrated in this repository or another one?

Hi,
RLBench is a set of environemnt, and so we wont ever put any baseline algorithms in this codebase.
If you are looking for baseline RL algortihms, then I suggest one of:
- [OpenAI Baselines ](https://github.com/openai/baselines)
- [OpenAI Spining Up](https://github.com/openai/spinningup)
- [RLLIB/RAY](https://github.com/ray-project/ray)

You can then use the algorithms there to train on these tasks :smile: 
If you are familiar with OpenAI Gym, then RLBench has recently become gym compatible. [See here](https://github.com/stepjam/RLBench#rlbench-gym).

So you’re saying there is no other RL repository which already has this one integrated? Correct me if I’m mistaken, but if there are no known, working, and integrated clients I feel like there will be a lot of debugging to do in this project and it may not be ready for use unless one is willing to go through debugging the whole API themselves. Im not trying to criticize, rather I’m asking to understand the scope of work if I were to use this project seriously. I have a paper & project which uses the vrep api and I’m figuring out the possibility of integrating that with this work https://github.com/jhu-lcsr/good_robot. 

> I'm looking into using RLBench, thanks for creating it! Are any baseline algorithms already integrated in this repository or another one?

Hi, I am working on adding some RL/IL algorithms upon this great environment for long term tasks. I am now making some training on Mujoco envs and will soon start my experiments on this env. I will keep updating related results on my repo once I have finished my tests on: id9502/RLFrame.git
Repository: stepjam/RLBench
Issue: Task gif link broken?

Seems the link going to the gifs of the task is not working anymore:

[https://drive.google.com/drive/folders/1TqbulbbCEqVBd6SBHatphFlUK2JQLkYu?usp=sharing](https://drive.google.com/drive/folders/1TqbulbbCEqVBd6SBHatphFlUK2JQLkYu?usp=sharing)

Hi, it appears that I am unable to access this link. Could you please share the most recent task gif link?

Is there a way to view these GIFs now ?
Repository: stepjam/RLBench
Issue: CoppeliaSim keeps showing simulation in progress, the Panda arm doesn't move,
Root cause: Algorithmic Error

Run tasks under `rlbench/tasks` using `task_builder`, start simulation with `+`, and `d` for demo, but CoppeliaSim keeps showing simulation in progress, the Panda arm doesn't move, and the simulation won't stop.
![image](https://github.com/user-attachments/assets/25686473-a31f-4b17-9591-a412c51c8eda)


Have you fixed it? I encounter the same issue when following the tutorial https://www.youtube.com/watch?v=bKaK_9O3v7Y&list=PLsffAlO5lBTRiBwnkw2-x0U7t6TrNCkfc. The panda arm doesn't move after typing start simulation with +, and d for demo.

By the way, you may stop the simulation by typing + again to exit.

This error will occur when using the master branch. After changing to add_demo_exec_action_to_misc, there will be no problem.


After testing, the code of version #220 in master can still display the demo normally, but the demo cannot be demonstrated in version #221, but the validator can still run normally.
Repository: stepjam/RLBench
Issue: Change simulation time step in demo collection
Root cause: Documentation Deficiency

Hi Stephen,

thanks for your continued work on RLBench. And great talk at ICRA this year!

Quick question: how to change the simulation time step for demo collection?

I've tried `env._scene.pyrep.set_simulation_timestep(XX)`, but that does not seem to work.

Best,
Jan
Repository: stepjam/RLBench
Issue:  Error: signal 11  after load libqxcb-glx-integration.so
Root cause: Environment Incompatibility

Thank you for your excellent work. I am testing on wsl2 under windows11, the system is ubuntu20.04. After installing CoppeliaSim and RLbench, do I need to configure the Qt environment? If so, what are the necessary configuration references?
I encountered the following problem, not sure what caused it, I would be very grateful if you can help me
![image](https://github.com/user-attachments/assets/b2da3928-91fb-4056-a937-35f76d6754e8)


did you @JYMiracle305 you solve the issue i am facing the same
Repository: stepjam/RLBench
Issue: Unable to correctly determine if the task was successfully executed
Root cause: Algorithmic Error

Hello, I have noticed that in RLBench, some tasks continue executing even after they are successfully completed because the simulator fails to correctly determine success. For example, in this case, the model has successfully placed two cups, but the program still does not recognize the success and continues running. Could you tell me how to solve this problem?
![image](https://github.com/user-attachments/assets/fd2aacda-6d12-4d49-b90f-30b450219c54)
Repository: stepjam/RLBench
Issue: NameError: name 'get_demo' is not defined
Root cause: Validation Gap

Hello,When I attempted to create a new task in RLbench, I encountered the following issue. Could you please advise me on how to resolve it? Thank you very much!
![微信图片_20241118231802](https://github.com/user-attachments/assets/23b0e425-71d9-4fc5-847f-36cc44779b05)
Repository: stepjam/RLBench
Issue: Can't use on headless machine
Root cause: Documentation Deficiency

Hello, when I config RLBench on a headless server, I try to use `nvidia-xconfig -a --use-display-device=None --virtual=1280x1024` as you say in README.md. However I find `bash: nvidia-xconfig: command not found`. I'm sure I can use `nvcc -V` and `nvidia-smi` on my machine. Can you give me some advice on fixing the problem. Thank you !

I met the same problem T-T

Do you solve the problem? It seems I need to reinstall the driver

Hi, had the same problem, the workaround we found was to define a virtual framebuffer with Xvfb, for example:

```sh
Xvfb :77 -screen 0 1024x768x16 &
```

And then run your program using this screen by setting a environment variable:

```sh
DISPLAY=:77 python program.py
```

Of course you could also export the variable first. Cheers

Thank you for your advices, now I can run without segmentation fault. However, similar issue as #259  happens, is it the problem of installation ? Thanks for any reply

I met the same problem TvT

I think Xvfb with correct env configuration is enough, if questions still holds, providing more details might be helpful !
Repository: stepjam/RLBench
Issue: How to add a camera and set its intrinsic and extrinsic parameters?
Root cause: Documentation Deficiency

I want to add a camera to the slide_block_to_target task, positioned near cam_front (rotating 30 degrees around cam_front with the base as the center). However, after adding the corresponding cam_front2 to all parts of RLBench related to cam_front, I still encounter the following error. How can I add a camera with the same intrinsic parameters but slightly different extrinsic parameters?
```
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/data_collection/record_demos.py", line 118, in <module>
    env.launch()
  File "/home/anaconda3/envs/lib/python3.11/site-packages/rlbench/environment.py", line 125, in launch
    self._scene = Scene(
                  ^^^^^^
  File "/home/anaconda3/envs/lib/python3.11/site-packages/rlbench/backend/scene.py", line 50, in __init__
    self._cam_front2 = VisionSensor('cam_front2')
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/anaconda3/envs/lib/python3.11/site-packages/pyrep/objects/vision_sensor.py", line 14, in __init__
    super().__init__(name_or_handle)
  File "/home/anaconda3/envs/lib/python3.11/site-packages/pyrep/objects/object.py", line 24, in __init__
    self._handle = sim.simGetObjectHandle(name_or_handle)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/anaconda3/envs/lib/python3.11/site-packages/pyrep/backend/sim.py", line 94, in simGetObjectHandle
    raise RuntimeError('Handle %s does not exist.' % objectName)
RuntimeError: Handle cam_front2 does not exist.
QObject::~QObject: Timers cannot be stopped from another thread
QMutex: destroying locked mutex
```

any answers?
Repository: stepjam/RLBench
Issue: ypeError: action() takes 3 positional arguments but 4 were given
Root cause: Interface Contract Mismatch

I can suceessfully run in PyRep, but when i run RLBench, it crashed. Why?

root@autodl-container-bf2c4487e6-930da208:/mnt/nas-a6000/hanyangyu/projects/robot/RLBench# python ./examples/single_task_rl.py
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
Reset Episode
['reach the red target', 'touch the red ball with the panda gripper', 'reach the red sphere']
[ 0.04562055 -0.0573288   0.04520147  0.05530244  0.0132626  -0.12701392
  0.03001259  1.        ]
Traceback (most recent call last):
  File "./examples/single_task_rl.py", line 42, in <module>
    obs, reward, terminate = task.step(action)
  File "/mnt/nas-a6000/hanyangyu/projects/robot/RLBench/rlbench/task_environment.py", line 99, in step
    self._action_mode.action(self._scene, action)
  File "/mnt/nas-a6000/hanyangyu/projects/robot/RLBench/rlbench/action_modes/action_mode.py", line 35, in action
    self.arm_action_mode.action(scene, arm_action, ignore_collisions)
TypeError: action() takes 3 positional arguments but 4 were given
QObject::~QObject: Timers cannot be stopped from another thread
QMutex: destroying locked mutex
QMutex: destroying locked mutex

xvfb-run -a python ./examples/imitation_learning.py
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
Traceback (most recent call last):
  File "./examples/imitation_learning.py", line 39, in <module>
    demos = np.array(demos).flatten()
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
QObject::~QObject: Timers cannot be stopped from another thread
QMutex: destroying locked mutex

I also meet the same problem when running examples, have you solved it yet ?

I fix this problem by modify line 35 in rlbench/action_modes/action_mode.py to self.arm_action_mode.action(scene, arm_action). It seems work now.
Repository: stepjam/RLBench
Issue: Print outputs are getting buffered
Root cause: External Dependency Breakage

Using the gym environment.

When working with print(), it seems that after calling env.reset(), print outputs are getting buffered i.e. not synchronously outputted to the console.
Invoking python with -u does not seem to make a difference.

Hi.
I think this is probably down to PyRep actually. Could you try something for me please? In PyRep, could you comment out the following 3 lines: [1](https://github.com/stepjam/PyRep/blob/master/pyrep/robots/arms/arm.py#L71), [2](https://github.com/stepjam/PyRep/blob/master/pyrep/robots/arms/arm.py#L172), [3](https://github.com/stepjam/PyRep/blob/master/pyrep/robots/arms/arm.py#L225). Remember to rerun the setup.py file for PyRep.

If your problem is resolved because of that, then I imagine that is the issue. The problem is, if you remove these lines, then you actually end up with a bunch of extra outputs from OMLP (the path planner behind the scenes). The fix would need to be in CoppeliaSim/V-REP.

I tried that and it seems to work just fine, so thank you very much! Not getting any ompl output, but then again I don't use any motion planning afaik
Repository: stepjam/RLBench
Issue: Task error?
Root cause: Interface Contract Mismatch

I run task builder and run any task py
Traceback (most recent call last):
  File "task_builder.py", line 134, in new_variation
    descriptions = self.scene.init_episode(
  File "/home/lemon/RLBench-master/tools/../rlbench/backend/scene.py", line 136, in init_episode
    self.task.validate()
  File "/home/lemon/RLBench-master/tools/../rlbench/backend/task.py", line 274, in validate
    self._waypoints = self._get_waypoints()
  File "/home/lemon/RLBench-master/tools/../rlbench/backend/task.py", line 415, in _get_waypoints
    feasible, way_i = self._feasible(waypoints)
  File "/home/lemon/RLBench-master/tools/../rlbench/backend/task.py", line 366, in _feasible
    path.set_to_end()
AttributeError: 'tuple' object has no attribute 'set_to_end'
Repository: stepjam/RLBench
Issue: Why are models trained on RLBench data generally suited for Keyframe Position Prediction?

Here are three prediction methods:

1. **Initial Frame Grasp Prediction**:
   - This method predicts the grasp point of the target object by analyzing the initial frame.

2. **Keyframe Position Prediction**:
   - This approach involves predicting the next keyframe position by inputting a series of keyframes, suitable for tasks that require consideration of intermediate state changes.

3. **Frame-by-Frame Action Prediction**:
   - In this method, each frame is inputted, and the next position is outputted, making it ideal for tasks that require continuous monitoring and real-time decision-making.

Why are models trained on RLBench data generally suited for Keyframe Position Prediction?
Repository: stepjam/RLBench
Issue: How to get Mask value of an object
Root cause: Documentation Deficiency

When collecting demo, i can get the one-channel mask image. It seems like each mask value corresponding to a specific object. But how I can I find the correspondence map, like 145 corresponding to the pencil?

The reply in https://github.com/stepjam/RLBench/issues/266#issuecomment-2649969548 is wrong. Since rlbence did not render rgb values for mask cameras. For those mask cameras, the points on the same object have the same value.

<img width="851" alt="Image" src="https://github.com/user-attachments/assets/fef93f8a-83a9-4dde-98c8-e1d241c2ee58" />

> The reply in [#266 (comment)](https://github.com/stepjam/RLBench/issues/266#issuecomment-2649969548) is wrong. Since rlbence did not render rgb values for mask cameras. For those mask cameras, the points on the same object have the same value.
> 
> <img alt="Image" width="851" src="https://private-user-images.githubusercontent.com/43157631/433724575-fef93f8a-83a9-4dde-98c8-e1d241c2ee58.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NDQ3MjM4NTYsIm5iZiI6MTc0NDcyMzU1NiwicGF0aCI6Ii80MzE1NzYzMS80MzM3MjQ1NzUtZmVmOTNmOGEtODNhOS00ZGRlLTk4YzgtZTFkMjQxYzJlZTU4LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA0MTUlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwNDE1VDEzMjU1NlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTY2MGY2MjA1N2U3MmQ0MWQ5ZmQwZjFjMmIxNTk4YjM1MjhjMDgwYzIwMGEyNzU4NTI4ODU0ZjhlYzVhOTAxYzEmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.W9USYD2PaTVttMHf_T6Uqtl1xMHsEUovA5ly6Ryuji4">

Yes, I sent the Voxposer simulation mask code before, that is my problem.
Repository: stepjam/RLBench
Issue: Bulb holder not visible in demonstrations generated for light_bulb_in
Root cause: Validation Gap

Is this a bug or am I missing something? Thanks for your help!

Task:

![Image](https://github.com/user-attachments/assets/8757bce3-aae0-4741-b2f8-df2bc2d02fbe)

Replay of collected Demonstration:

https://github.com/user-attachments/assets/62860321-d509-418a-91da-c87586367251
Repository: stepjam/RLBench
Issue: OpenWineBottle task doesn't initialize correctly after the first demo
Root cause: Algorithmic Error

I was collecting multiple demos for the OpenWineBottle task and noticed that it works fine on the first demo collected (cap is on bottle and robot removes cap from bottle), but on subsequent attempts the bottle is initialized with the cap on the table instead of the bottle, and the robot tries to plan its motion to the top of the bottle and then the environment just resets and puts the bottle in a different location. I've attached an image of the environment with the cap on the table, which is how it keeps being initialized after the first recorded demo.
![Screenshot from 2020-01-10 09-09-57](https://user-images.githubusercontent.com/22223086/72168296-1a6bd780-338a-11ea-8905-474c16f39924.png)
Repository: stepjam/RLBench
Issue: RuntimeError: Handle cam_over_shoulder_left_mask does not exist.
Root cause: Environment Incompatibility

when I running "python task_builder.py",I got those error:
###############################################
Traceback (most recent call last):
  File "task_builder.py", line 289, in <module>
    scene = Scene(pr, robot, obs_config)
  File "/home/xbh/RLBench/tools/../rlbench/backend/scene.py", line 50, in __init__
    self._cam_over_shoulder_left_mask = VisionSensor(
  File "/home/xbh/.local/lib/python3.8/site-packages/pyrep/objects/vision_sensor.py", line 14, in __init__
    super().__init__(name_or_handle)
  File "/home/xbh/.local/lib/python3.8/site-packages/pyrep/objects/object.py", line 24, in __init__
    self._handle = sim.simGetObjectHandle(name_or_handle)
  File "/home/xbh/.local/lib/python3.8/site-packages/pyrep/backend/sim.py", line 94, in simGetObjectHandle
    raise RuntimeError('Handle %s does not exist.' % objectName)
RuntimeError: Handle cam_over_shoulder_left_mask does not exist.
QObject::~QObject: Timers cannot be stopped from another thread
QMutex: destroying locked mutex
#############################################
what should I do?

To me, I reinstalled the RLBench.

> To me, I reinstalled the RLBench.

@joonsu0109gh, I tried reinstalling PyRep and RLBench using `pip3 install .`. I still get the error despite this change.  Did you clear any caches or take any additional steps?

I mean, try removing the RLBench directory and reinstalling it by following the installation guide from this work. I hope this works for you too.

Thank you for the suggestion. I guess it was an issue with the specific commit that I was working on. I switched to `master` and reinstalled RLBench again, and now it is working again! 

Perhaps the issue is saving changes in [Coppeliasim's GUI](https://github.com/Boxjod/RLBench_ACT#:~:text=Note%3A%20Do%20not,the%20following%20commands%3A). It can apparently result in missing components.
Repository: stepjam/RLBench
Issue: swap_arm.py producing Error
Root cause: Unimplemented Feature Gap

Hi,

I've tried changing the arm in swap_arm.py from "sawyer" to "jaco" and "mico", but both result in the following error:

python swap_arm.py  
Reset Episode  
Traceback (most recent call last):  
  File "/home/-/genima/rlbench/examples/swap_arm.py", line 43, in <module>  
    descriptions, obs = task.reset()  
  File "/home/-/genima/rlbench/rlbench/task_environment.py", line 79, in reset  
    desc = self._scene.init_episode(  
  File "/home/-/genima/rlbench/rlbench/backend/scene.py", line 136, in init_episode  
    self.task.validate()  
  File "/home/-/genima/rlbench/rlbench/backend/task.py", line 274, in validate  
    self._waypoints = self._get_waypoints()  
  File "/home/-/genima/rlbench/rlbench/backend/task.py", line 409, in _get_waypoints  
    feasible, way_i = self._feasible(waypoints)  
  File "/home/-/genima/rlbench/rlbench/backend/task.py", line 360, in _feasible  
    path = point.get_path(ignore_collisions=True)  
  File "/home/-/genima/rlbench/rlbench/backend/waypoints.py", line 55, in get_path  
    path = arm.get_path(self._waypoint.get_position(),  
  File "/home/-/.pyenv/versions/3.10.12/lib/python3.10/site-packages/pyrep/robots/arms/arm.py", line 453, in get_path  
    p = self.get_nonlinear_path(  
  File "/home/-/.pyenv/versions/3.10.12/lib/python3.10/site-packages/pyrep/robots/arms/arm.py", line 388, in get_nonlinear_path  
    configs = self.solve_ik_via_sampling(  
  File "/home/-/.pyenv/versions/3.10.12/lib/python3.10/site-packages/pyrep/robots/arms/arm.py", line 146, in solve_ik_via_sampling  
    config = sim.simGetConfigForTipPose(  
  File "/home/-/.pyenv/versions/3.10.12/lib/python3.10/site-packages/pyrep/backend/sim.py", line 1462, in simGetConfigForTipPose  
    _check_return(ret)  
  File "/home/-/.pyenv/versions/3.10.12/lib/python3.10/site-packages/pyrep/backend/sim.py", line 27, in _check_return  
    raise RuntimeError(  
RuntimeError: The call failed on the V-REP side. Return value: -1  

I also tried changing the arm in dataset_generator.py to "jaco", but that didn't work either. Do you have supported tasks for other arms anywhere listed?

Any help would be appreciated. Thanks!
Repository: stepjam/RLBench
Issue: Get end-effector pose from path planner
Root cause: Unimplemented Feature Gap

Thanks for your great work!

In `rlbench.action_modes.arm_action_modes.py`, `class EndEffectorPoseViaPlanning`, in the `self.action` function, it calls `scene.robot.arm.get_path` to generate a path from current pose to the target pose. The returned path is in joint space. But I now want the end-effector trajectory, so I would like to know how can I directly get the path from the planner in world coordinate space. 

I know that I can call `scene.get_observation()` after each `scene.step()` to get the end effector pose. But I currently need a path directly from the planner, not from the execution results. So is there any way to get that?

Thanks a lot for your reply!

<img width="948" alt="Image" src="https://github.com/user-attachments/assets/831c702b-113f-4faf-aecd-10ff2093d857" />
Hi, dosesn't the data_generator directly output that?

Which data_generator you are referring to? Those data (joint angle, velocity, ee pose, etc) are available in the `Observation` class, but it has to be obtained after rollout. For getting the end-effector pose before rollout, I didn't find any code that can do that.