zhiminy/EvalEng huggingface dataset:
Repository: stepjam/RLBench
Issue: Bug in dataset_generator.py while saving joint_velocities
Root cause: Algorithmic Error

Hi!

Thanks for the great work!

I would like to point out a bug in dataset_generator.py that may happen while saving joint_velocities which generates duplicate frames with exactly the same joint_positions and joint_velocities.

I downloaded the generated demos from PreACT. And while I was inspecting the demo structure, I found that the joint_velocities can be calculated by joint_positions of adjacent frames. I manually confirmed that it's mostly correct: difference between joint_positions divided by delta time--0.05s is roughly equal to joint_velocities.

However, I notice that there are certain frames that have exactly the same joint_positions and joint_velocities. The same joint_positions indicates that the robot did not move, while the joint_velocities could indicate some bug?

Here is what I found from from the pkl file of this demo (val/close_jar/all_variations/episodes/episode0/low_dim_obs.pkl):


Velocity error analysis:
Mean error per joint: [0.00015695 0.001749   0.00015985 0.00051234 0.00014436 0.00172141
 0.00058723]
 Max error per joint: [0.01520634 0.13777256 0.01447678 0.03603935 0.01175404 0.14060974
 0.08033276]
 Worst frame index: 115
 Worst frame error: 0.1406097412109375
 Significant mismatch found! Check if demo is corrupted or delta_time is wrong.
joint_positions[115] = [-0.18113899  0.50589633  0.01586798 -1.98203373 -0.01134741  2.41657305
  0.91748607]
joint_positions[116] = [-0.18113899  0.50589633  0.01586798 -1.98203373 -0.01134741  2.41657305
  0.91748607]
joint_velocities[116] = [-0.00416279  0.13777256 -0.00335693 -0.00947475 -0.00200748  0.14060974
 -0.00582695]
joint_velocities[115] = [-0.00416279  0.13777256 -0.00335693 -0.00947475 -0.00200748  0.14060974
 -0.00582695]
Estimated v = [0. 0. 0. 0. 0. 0. 0.]


Links can be found here: 
 
PreACT repo: https://github.com/peract/peract#download
PreACT saved data on Huggingface: https://huggingface.co/datasets/hqfang/RLBench-18-Tasks

Best,
Xianjie



Not sure whether it's because the demo is generated by a relatively old version of RLBench and the bug has been fixed in the new version?

Similarly, there are two frames in episode1 that have the same velocities and positions:

joint_positions[130] = [-0.09380674 -0.08350965 -0.11569616 -2.88443708 -0.02401292  2.73060584
  0.52143824]
joint_positions[131] = [-0.09380674 -0.08350965 -0.11569616 -2.88443708 -0.02401292  2.73060584
  0.52143824]
joint_velocities[130] = [ 0.00741005  0.12661457 -0.00412464 -0.00038147  0.02579689  0.12641907
 -0.02010822]
joint_velocities[131] = [ 0.00741005  0.12661457 -0.00412464 -0.00038147  0.02579689  0.12641907
 -0.02010822]

any news on this?
Repository: stepjam/RLBench
Issue: CoppeliaSim Probelm
Root cause: Environment Incompatibility

When I follow the tutorial to run the CoppeliaSim virtual environment, it seems like I'm missing something. As shown in the image below, I don't have options like 'waypoint'.

![Image](https://github.com/user-attachments/assets/85fc246d-a328-4c7a-bb94-a9c4408a9970)

I would be extremely grateful if I could get a reply to help solve this issue.
Repository: stepjam/RLBench
Issue: About save probelm
Root cause: Resource Mishandling

Thanks for your great work. I want to change the existing task and save it. But I met some problems. After duplicating/copying the task and renaming it, I can't open it.
'''
The python file will be reloaded when simulation is restarted.
(q) to quit.
(!) to run task validator.
(+) run the simulator
(n) for new task.
(s) to save the .ttm
(r) to rename the task
(u) to duplicate/copy the task
s
Traceback (most recent call last):
  File "task_builder.py", line 348, in <module>
    loaded_task.save_task()
  File "task_builder.py", line 168, in save_task
    self.task.get_base().save_model(ttm_path)
  File "/home/yutian/miniconda3/envs/rlbench/lib/python3.8/site-packages/pyrep/objects/object.py", line 643, in save_model
    sim.simSaveModel(self._handle, path)
  File "/home/yutian/miniconda3/envs/rlbench/lib/python3.8/site-packages/pyrep/backend/sim.py", line 357, in simSaveModel
    _check_return(val)
  File "/home/yutian/miniconda3/envs/rlbench/lib/python3.8/site-packages/pyrep/backend/sim.py", line 27, in _check_return
    raise RuntimeError(
RuntimeError: The call failed on the V-REP side. Return value: -1
QObject::~QObject: Timers cannot be stopped from another thread
QMutex: destroying locked mutex
'''


have you resolved the problem？
Repository: stepjam/RLBench
Issue: Rendering artifact: black rectangle appears in scene
Root cause: Validation Gap

While using RLBench to collect data with custom camera views, I encountered a persistent black rectangular artifact in the rendered images. The black shape appears to float in the scene and changes relative position depending on the camera pose. As shown in the image below, the black artifact is located near the corner of the wall. And in the depth map, this object appears as yellow.  It seems to be a rendering bug. Is there a proper way to disable or remove it without affecting the simulation?

![Image](https://github.com/user-attachments/assets/0c21615f-3e9c-4dfe-85be-ee1088652b14)
Repository: stepjam/RLBench
Issue: Reproduce saved demo in simulator again
Root cause: Documentation Deficiency

Hello, thanks for your work. Is there the possibility to exactly resimulate in the simulator a given demo generated with the default `rlbench/dataset_generator.py`? How to do it?

You can set live_demos=False, and set task_env dataset_root as well, I reproduce one successful just now

@iamstupidd can you share the code?

Simple code is,
`    task_env._dataset_root = "<your_path>"
    demo, = task_env.get_demos(amount=1, live_demos=False)`
Alternative for reproduce from the very beginning,
`   obs_dict =task_env.reset_to_demo(demo)`



@iamstupidd okay and then how to execute the loaded demo action by action?
Repository: stepjam/RLBench
Issue: Tasks with button presses never complete successfully
Root cause: Validation Gap

I have tried collecting demos in the TvOn, TvOff, and ChangeChannel tasks, as well as PushButton and PushButtons, but none of them complete successfully. They all get initialized correctly and the robot plans and executes the motions, but it seems the actual button presses are maybe not being registered? The environment just keeps resetting and executing the task without success.

Thanks for reporting. Will take a look at this later today.

I was able to run all of the tasks except LampOn, that one still seems to not respond to a button press.

Thanks. This is now resolved :smile:.
Repository: stepjam/RLBench
Issue: How to run RLBench headless and render on a remote server without sudo.
Root cause: Documentation Deficiency

Please ask how to config and run RLBench headless and render on a remote server without sudo authority.

<img width="1484" height="619" alt="Image" src="https://github.com/user-attachments/assets/2855cdc6-eef9-4f52-a112-3566bb0af78c" />

> Please ask how to config and run RLBench headless and render on a remote server without sudo authority.
> 
> <img alt="Image" width="1484" height="619" src="https://private-user-images.githubusercontent.com/72130560/466088204-2855cdc6-eef9-4f52-a112-3566bb0af78c.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTYxMTQ3MjYsIm5iZiI6MTc1NjExNDQyNiwicGF0aCI6Ii83MjEzMDU2MC80NjYwODgyMDQtMjg1NWNkYzYtZWVmOS00ZjUyLWExMTItMzU2NmJiMGFmNzhjLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA4MjUlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwODI1VDA5MzM0NlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWZjMGZlOWViZmEwZjdlYzBlMmJjODM3NmEyMTZiNGJlYjVjNzg2MTI2NzY2ODJkMmQxYzgzOWE0ZWMzZmZhMmImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.PmsvfvB8uueYJBC5Xu06tqPjk4tuqgkDVjrwNDIAJbs">

how to solve this ?


You can use `xvfb-run -a python your_rlbench_program.py` to run it on a headless server. This has a small (30% in my experience) performance penalty compared to a hardware accelerated X server (but to start an X server you need sudo).
Repository: stepjam/RLBench
Issue: Non linear path determinism
Root cause: Unimplemented Feature Gap

Hello, I am having an issue with reproducibility. Even when fixing all the python (and libraries) related seeds, I get different results between different runs of the same seed in RLBench/PyRep. This is apparently due to the intrinsic randomness in the PyRep's Arm get_nonlinear_path method, which every time is computing a different path even if the python's seeds are the same, and this is causing the objects in the scene to end in slightly different positions. Do you know how I can avoid this randomness by any chance? Lowering the distance_threshold parameter is not solving the problem.
Thanks!
Repository: stepjam/RLBench
Issue: Code stuck in 'sensor.handle_explicitly()'
Root cause: Resource Mishandling

When I run the eval code in sam2act, I found my code stuck in this function. How could I solve this issue?
<img width="1498" height="401" alt="Image" src="https://github.com/user-attachments/assets/e738fbeb-9abd-445a-bdcb-6739b21bedbf" />

<img width="1019" height="214" alt="Image" src="https://github.com/user-attachments/assets/a4003cfa-4308-49c5-871c-f938338a3235" />
Repository: stepjam/RLBench
Issue: Stack cups ambiguous instruction.
Root cause: Validation Gap

Hi, I think one of the possible language instructions in stack_cup is incorrect.
The task is to place two cups into one specified cup. But one of the language variations is "place two of the cups onto the odd cup out", but there is no way to determine which cup is the odd cup out.


Looking at the code with 20 colors available:
```
  # Cup2 gets the target color (based on variation index)
  self.cup2_visual.set_color(target_rgb)

  # Cup1 gets ANY color except the target (19 possibilities)
  random_idx = np.random.choice(len(colors))
  while random_idx == index:
      random_idx = np.random.choice(len(colors))
  _, other1_rgb = colors[random_idx]

  # Cup3 gets ANY color except the target (19 possibilities)
  random_idx = np.random.choice(len(colors))
  while random_idx == index:
      random_idx = np.random.choice(len(colors))
  _, other2_rgb = colors[random_idx]

  # Cup placement logic is identical:
  self.boundary.sample(self.cup2, min_distance=0.05,
                             min_rotation=(0, 0, 0), max_rotation=(0, 0, 0))
  self.boundary.sample(self.cup1, min_distance=0.05,
                             min_rotation=(0, 0, 0), max_rotation=(0, 0, 0))
  self.boundary.sample(self.cup3, min_distance=0.05,
                             min_rotation=(0, 0, 0), max_rotation=(0, 0, 0))
```


  The issue:
  - With 20 colors, there's only a ~5.3% chance (1/19) that cup1 and cup3 get the same color
  - In ~95% of episodes, all three cups have different colors
  - When all three are different, the instruction "place two of the cups onto the odd cup out" is ambiguous. There is no objective "odd one out"


Should we remove this instruction?

This is actually only relevant for https://github.com/robot-colosseum/robot-colosseum, not RLbench itself.
Repository: stepjam/RLBench
Issue: How can I run RLBench without relying on a graphical interface (including virtual screen)?
Root cause: Architectural Constraint

I need to run RLBench tests on a remote cluster (CentOS) without a graphical interface, but I can't install and use Xvfb on the cluster due to insufficient permissions.
Is there a way to make RLBench run completely without relying on a (virtual) screen?

As far as I am aware you cannot. There must be some OpenGL GLX context running, either an X server or virtual X server through xvfb.
You could try building the xvfb binary on the cluster, to use it without installing through yum, because you do not need sudo to run it. Otherwise, take a look at using a apptainer container on your cluster.
Repository: stepjam/RLBench
Issue: EmptyContainer task crashes with FileNotFoundError
Root cause: Environment Incompatibility

When I run the EmptyContainer task, I get the error:
```
Traceback (most recent call last):
  File "collect_rlbench_data.py", line 192, in <module>
    collect_demos(env, tasks, h5_root, args.n_demos_per_task, len(tasks))
  File "collect_rlbench_data.py", line 103, in collect_demos
    demo = task.get_demos(1, live_demos=True)[0]
  File "/home/adam/venv3/lib/python3.5/site-packages/rlbench/task_environment.py", line 239, in get_demos
    demos = self._get_live_demos(amount)
  File "/home/adam/venv3/lib/python3.5/site-packages/rlbench/task_environment.py", line 248, in _get_live_demos
    self.reset()
  File "/home/adam/venv3/lib/python3.5/site-packages/rlbench/task_environment.py", line 71, in reset
    randomly_place=not self._static_positions)
  File "/home/adam/venv3/lib/python3.5/site-packages/rlbench/backend/scene.py", line 93, in init_episode
    descriptions = self._active_task.init_episode(index)
  File "/home/adam/venv3/lib/python3.5/site-packages/rlbench/tasks/empty_container.py", line 36, in init_episode
    self.bin_objects = sample_procedural_objects(self.get_base(), 3)
  File "/home/adam/venv3/lib/python3.5/site-packages/rlbench/backend/task_utils.py", line 10, in sample_procedural_objects
    os.listdir(assets_dir), num_samples, replace=False)
FileNotFoundError: [Errno 2] No such file or directory: '/home/adam/venv3/lib/python3.5/site-packages/rlbench/backend/../assets/procedural_objects'
QObject::~QObject: Timers cannot be stopped from another thread
QMutex: destroying locked mutex
```

Hi Adam,
This is strange as the CI server doesn't catch this. I'm wondering if it's perhaps something to do with the  virtual env :thinking:. In the setup.py script, the assets directory is included for install. Could you perhaps try running that task outside a venv and see if you still get an issue?

I was able to reproduce this. Fix on the way.
Repository: stepjam/RLBench
Issue: Segmentation mask for wrist camera not tracking with RGB/Depth
Root cause: Algorithmic Error

The segmentation mask for the wrist camera seems to stay where it was initialized, instead of tracking with the movement of the hand. This can be seen in the GIF below that shows the wrist RGB and wrist segmentation mask side-by-side synchronized in time. I'll emphasize that the segmentation mask is **not** from one of the shoulder cams, it actually is associated with the wrist camera.

![Peek 2020-01-11 12-43](https://user-images.githubusercontent.com/22223086/72209836-321d8b80-3470-11ea-9448-419a0077d388.gif)


Hi Adam,
Thanks for pointing out. I can resolve this later today :)
Repository: stepjam/RLBench
Issue: Add videos of each task

This is a feature request. I think it would be nice to have a way of viewing an execution of each task without actually having to load up the simulator and run it. Your YouTube video that gives an overview of RLBench is very nice, but I'm still finding that I see a task name and, though it's descriptive, I ideally could watch a video of it so I know exactly what the robot is going to try to do. 

If this is impractical then you can close, but a couple thoughts are having a readme page with GIFs that show each task, or having YouTube videos of each task that you can just click to and watch the video.

My current use case is I've recorded demos for every task in a headless manner. I'm now trying to divide the data into train/test sets so I know if they were seen/unseen environments in testing, but it's hard for me to tell without actually launching the simulator or trying to see if it's shown in your overview YouTube video.

Yes, I think this is a good idea. hopefully will update this issue in a few days :smile:. 

I made some GIFs of each task for my own purposes, figured I could share them with you. Has RGB feed for each task from all three camera views, I left it in a flat structure so you can just click through them one by one and see them right in Google Drive: https://drive.google.com/drive/folders/1TqbulbbCEqVBd6SBHatphFlUK2JQLkYu?usp=sharing

Ohh, that's very helpful. Thanks!
Apologies for not getting around to doing it myself yet; have been prioritising bugs. I can add a link to this in the readme :)

@adamconkey do you happen to still have the GIFs? The gdrive link has expired.

Is there a way to view these GIFs now?
Repository: stepjam/RLBench
Issue: Example RL algorithms or other repos utilizing RLBench?
Root cause: Documentation Deficiency

So you’re saying there is no other RL repository which already has this one integrated? Correct me if I’m mistaken, but if there are no known, working, and integrated clients I feel like there will be a lot of debugging to do in this project and it may not be ready for use unless one is willing to go through debugging the whole API themselves. I'm not trying to criticize, rather I’m asking to understand the scope of work if I were to use this project seriously. I have a paper & project which uses the vrep api and I’m figuring out the possibility of integrating that with this work https://github.com/jhu-lcsr/good_robot.

Sorry for duplicating https://github.com/stepjam/RLBench/issues/25, but my followup question seems to have been lost, and I can't figure out how to reopen it.

I'm not sure what you mean by a repository that 'integrates' this one. RLBench is an environment, and so it can be used with any reinforcement learning repository (e.g. [RLzoo](https://github.com/tensorlayer/RLzoo)). The repo is well tested both through unit tests, task integration tests, and manual use through myself and others who use this project; of course bugs will emerge over time, but we are continuing to improve.

Do you know of any public pytorch code which already supports RLBench as an environment?

I can see [RLzoo](https://github.com/tensorlayer/RLzoo/blob/ea5a5734e7a9a9d9610b7405ec1954302cb80c83/rlzoo/common/build_rlbench_env.py), which you suggested, does have integrated support, but it uses TF.

Hi,
Not that I am aware of, but any framework that 'supports' OpenAI Gym will automatically be compatible with RLBench.
Repository: stepjam/RLBench
Issue: task_ttms not found
Root cause: Configuration Error

The latest changes to setup.py apparently doesn't copy the task_ttms to site-packages, and then running the imitation_learning.py cannot find the task ttm file and fails. I'm using python 3.6.9.

Thanks for pointing out 😄 

Thanks for the quick response :)

-Nima


On Sun, Jan 19, 2020 at 1:37 PM Stephen James <notifications@github.com>
wrote:

> Thanks for pointing out 😄
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/stepjam/RLBench/issues/34?email_source=notifications&email_token=AAJR4MHO4A6GSM5UHFGO6CTQ6TB27A5CNFSM4KI22IR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJK5IBA#issuecomment-576050180>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAJR4MAOHOS66TS7IK46KILQ6TB27ANCNFSM4KI22IRQ>
> .
>
Repository: stepjam/RLBench
Issue: low_dim_obs.pkl too large
Root cause: Algorithmic Error

I noticed when using dataset_generator.py that the low_dim_obs.pkl files are around 20MB each! I looked around a bit, and I think this might be because you don't None the *mask* images in obs before pickling.

Thanks! Yes, currently the masks are not getting saved as images... I'll get a fix out for this later today :)
Repository: stepjam/RLBench
Issue: Render speed penalty
Root cause: Documentation Deficiency

- Using the gym api

when calling render, I'm getting a penalty of 30x on my step time. Is this to be expected? Are there ways to improve step time while rendering? Like, for example, speeding up simulation time or something?

I'm curious, because I'm currently runinng experiments where I want to inspect the robot at different time steps but I don't know when these appear beforehand. One solution would be recording a video of every time step but given the insane slow down I consider this infeasible. 
As an alternative I thought of recording all actions which would allow me to "replay" certain parts of the simulation. For that to work I would need to be able to restore the simulator state for arbitrary time points. Is this possible and if yes what is the suggested approach for this?

Hi timody,

Yes this penalty is to be expected. There's always going to be a penalty when calling the rendering, and yes, as you point out ideally you should be calling this only when needed. Alas, you have a few options:
- You can decrease the size of the images by modifying [this line](https://github.com/stepjam/RLBench/blob/master/rlbench/gym/rlbench_env.py#L73).
- You can swap to using the `RenderMode.EXTERNAL_WINDOWED` rather than the `RenderMode.OPENGL3_WINDOWED` [here](https://github.com/stepjam/RLBench/blob/master/rlbench/gym/rlbench_env.py#L75). The former wont have any shadows, and so it will be faster to render.
- Ensure that you are using the GPU to render. Beware, if you use something like Xvfb, then it **does not use hardware rendering**, so this will dramatically slow down your rendering. 
- As you pointed out, you could replay the trajectories later. I actually plan to include support for 'reset to demonstrations' which would allow you to reset to a specific point in the demo. However, it is not supper high on the priority list at the moment. 

Best,
Stephen

Hi Stephen,

thanks for the tips! I'm going to try decreasing the image size and switch the render mode, as I don't need shadows for my purposes.
I'm aware of the issue with virtual frame buffers, thank you. Do you know how I can make sure that coppelia sim is using the gpu to render?
About replaying trajectories: what do you mean with demonstrations? If you give me a few pointers I could try making a pull request.

Best
Julius

I tried your suggestions: switching the render mode yields a noticeable difference in appearance (no shadows) and improves render time. Yay! Changing the resolutions parameters, however, does not seem to actually change the resolution of the gym cam.

hello sir, I have the same problem about the rendering speed, so how can I switch to use GPU to render?

I'm assuming you mean when you run headless? This is related directly to CoppelisSim, and is not currently supported, you can track the issue here: https://github.com/stepjam/PyRep/issues/150

You can now do GPU rendering: https://github.com/stepjam/RLBench/blob/master/README.md#running-headless