zhiminy/EvalEng huggingface dataset:
Repository: stepjam/RLBench
Issue: Baseline Imitation Learning Policies & Results
Root cause: Documentation Deficiency

I really love this incredible, amazing effort; just to get a sense of using this library fordeveloping new strategies for imitation learning & reinforcement learning from vision in diverse, multi-task environments, it would be nice to have some benchmarks (or at least brief experimental writeups) to build off of!

Specifically, it would be really nice to get a sense of a "base" policy architecture, amount of demonstration data, and expected success rate for a subset (if not all) of the tasks in the benchmark.

I'm happy to help with running things out for other environments if there's code available, but if there's at least a writeup available, I'd really love to see it! Thanks!

Hi @siddk ,
Yes I agree that it would be nice. The reason why one doesn't exists is because simple observation -> joint velocity policies didn't work well because the tasks are challenging and sparsely rewarded, and so more specialized approaches are needed, e.g. my own [C2F-ARM](https://arxiv.org/abs/2106.12534).
However, I have recently added shaped rewards for some tasks, and so maybe now would be a good time for me to put up some starter code for learning RL/BC policies in pytorch. I'll try and find some time over the next month or two. 

@stepjam - I can totally see how that’s the case for RL? When it comes to imitation learning/BC - did you find similar results (even as you scaled up the number of demonstrations)? 

Hey @stepjam - just wanted to follow-up on this (no big rush); any chance of getting a simple BC agent that works for some number of pre-generated demonstrations?
Repository: stepjam/RLBench
Issue: Issue while running rlbench_gym.py
Root cause: Environment Incompatibility

I am getting this issue when running examples in `RLBench/examples/rlbench_gym.py`

`QXcbIntegration: Cannot create platform OpenGL context, neither GLX nor EGL are enabled
terminate called after throwing an instance of 'std::logic_error'
  what():  basic_string::_M_construct null not valid
Aborted (core dumped)`

**I am not getting this error while running other examples in RLBench.**

Any help would be appreciated! Thanks.
Repository: stepjam/RLBench
Issue: Adding a new robot into RLBench
Root cause: Documentation Deficiency

Hello!, like you said in the swapping arms section: "Don't see the arm that you want to use? Your first step is to make sure it is in PyRep, and if not, then you can follow the instructions for importing new arm on the PyRep GitHub page. After that, feel free to open an issue and we can being it in to RLBench for you". I already imported a new arm to PyRep, I made a pull request but I don't have any answer yet. I need to bring that arm to RLBench, Could you help me please?. Now, I am working on importing the entire robot into PyRep, so in the future I will need help again. Thanks in advance. 

Hi,
 Do you fix the problem? Can you tell us how to do this? How to import fetch robot in RLBench?

Thanks.

@Bailey-24 
Hello,
Have you successfully imported the robot? I also want to try to import it, but I am not sure whether it will be successful？
Thanks.
Repository: stepjam/RLBench
Issue: Bug Report. gym wrapper returns np._bool causes error when using stable-baselines3
Root cause: Interface Contract Mismatch

I was trying to connect RLBench with stable-baselines3 and found a minor error.
I used the code from #103, and changed state to vision.
  ```
import gym
import rlbench.gym
import stable_baselines3.common.env_checker
from stable_baselines3 import PPO


env = gym.make('reach_target-vision-v0')

print(stable_baselines3.common.env_checker.check_env(env))
model = PPO('MlpPolicy', env, verbose=1)
model.learn(total_timesteps=10000)

obs = env.reset()
for i in range(1000):
    action, _states = model.predict(obs, deterministic=True)
    obs, reward, done, info = env.step(action)
    print(obs.shape)
    print(reward)
    if done:
      obs = env.reset()

env.close()
  ```

When I run this code, such error occured from stable-baselines3 env checker:
  ```
  assert isinstance(done, bool), "The `done` signal must be a boolean"
AssertionError: The `done` signal must be a boolean
```

This was due to the incompatibility bewteen np._bool and bool. In rlbench/gym/rlbench.env.py line107, `terminate` is a np._bool type, which makes `isinstance(done, bool)` False. To fix it, I simply typecasted and it works:

```
def step(self, action) -> Tuple[Dict[str, np.ndarray], float, bool, dict]: 
        obs, reward, terminate = self.task.step(action) 
        terminate = bool(terminate) 
        return self._extract_obs(obs), reward, terminate, {} 

```
Repository: stepjam/RLBench
Issue: Is there a Dockerfile for running RLBench?
Root cause: Documentation Deficiency

I am having issues in setting up headless rendering in a server without sudo access. Is there a Dockerfile available for running RLBench? 

Thanks! 

Same here

You can use docker image with coppeliasim and pyrep [link](https://hub.docker.com/r/mingupingu/coppeliasim_pyrep), then install RLBench python package

great help, thx
Repository: stepjam/RLBench
Issue: Handle Mico_target does not exist
Root cause: Documentation Deficiency

I just want to replace the Panda arm with Mico arm, I just simply edited the task.ttt where I changed the Panda arm to Mico arm. Then I edited the code: 

    self._robot = Robot(Mico(), MicoGripper())

And when I run the demo script I got the error message.

So is there anybody have success experience to replace the arm ?
Or is there some detailed descriptions for changing arm?

I plan to make it much easier to swap out the arm. Please bear with me on this. Thanks :)
Repository: stepjam/RLBench
Issue: Question: Understanding the "task description" given by init_episode() for each task.py

Hi,

I'm new to RLbench and want to do research on language-conditioned imitation learning using RLbench.

Could someone helping me to confirm my understanding of the task description returned by each init_episode() function?
Take [straighten_rope.py](https://github.com/stepjam/RLBench/blob/master/rlbench/tasks/straighten_rope.py) as an example, these instructions are returned 
`['straighten rope',
                'pull the rope straight',
                'grasping each end of the rope in turn, leave the rope straight'
                ' on the table',
                'pull each end of the rope until it is straight',
                'tighten the rope',
                'pull the rope tight']`
I found that reset method of [task_environment.py](https://github.com/stepjam/RLBench/blob/master/rlbench/task_environment.py) make use of init_episode() so that the initial observation and a set of sentences (I listed above) will be returned to the agent at timestep t = 0.

Does this mean a language-conditioned policy have to conditioned on "straighten ..., grasping ..., on the table, ...., pull the rope tight" at once starting from t = 0 ?  Naively, one could summarize all sentences into one big vector and let policy condition it.

Or the policy is first conditions on "straighten rope", then at some point during evaluation, we receives "pull the rope straight" and after this sub-goal is done, the agent receives new instruction "grasping each end of the rope in turn, leave the rope straight" and we keep going until all sub-goal (instruction in the list) are consumed?

I personally prefer the first setting so that the agent is expose to all instructions and is responsible to figure out how to follow it on its own instead of receiving new instructions after each sub-goal of the task is completed. 
I found lots of other benchmark have the 2nd setting which is the main reason I want to use RLbench for my research!

Thank you very much 
 

Problem solved!

I notice that those task descriptions have overlap meanings!
Repository: stepjam/RLBench
Issue: [TYPO] potential typo in task desc of straighten_rope.py

Hi,

I guess the "'pull each end of the rope until **is** is straight'" should be "'pull each end of the rope until **it** is straight'" ?
the file[ link](https://github.com/stepjam/RLBench/blob/master/rlbench/tasks/straighten_rope.py) its here.

Thank you
Repository: stepjam/RLBench
Issue: [TYPO --> BUGS??] dataset_generator.py get task descriptions wrong?
Root cause: Interface Contract Mismatch

Hi,

in the file [dataset_generator.py](https://github.com/stepjam/RLBench/blob/master/tools/dataset_generator.py) , line 246, it has `obs, descriptions = task_env.reset()` to retrieve text descriptions. 

However, in [task_environment.py](https://github.com/stepjam/RLBench/blob/master/rlbench/task_environment.py) line 89 of reset(), it returns the following: `return desc, self._scene.get_observation()` .

It seems like in dataset_generator what is restored as "descriptions" is actually observation. 
Should it be `descriptions , obs = task_env.reset()` ???


same question
Repository: stepjam/RLBench
Issue: errors using UR10 for imitation_learning.py
Root cause: Unimplemented Feature Gap

Hi there, I am trying to use UR10 collecting data for imitation learning using imitation_learning.py file. I am using the UR10 model with robotiq85 from pyrep. It works to some degree, however, the pose of UR10 to reach the target is weird (different from the ur5) as you can see from the pictures.  I checked there is no setup difference between the UR5 under the rlbench/robot_ttms and my built UR10. what might be the issues? 

![Screenshot from 2023-02-12 16-09-59](https://user-images.githubusercontent.com/35995716/218322563-0d5b442e-d102-46e2-91d0-b4eb2cfc8622.png)
![Screenshot from 2023-02-12 16-10-34](https://user-images.githubusercontent.com/35995716/218322588-346f52c3-2a0e-4314-81a9-450e3047e6fe.png)

thanks.
Repository: stepjam/RLBench
Issue: Error running task_builder.py
Root cause: Environment Incompatibility

Hi there, I'm trying to initiate a new task by running task_builder.py following the tutorials, and I met this kind of issue:
![image](https://user-images.githubusercontent.com/53253395/222077086-7fe91a7e-bf32-499b-a7ea-117127fa2d27.png)
![image](https://user-images.githubusercontent.com/53253395/222077473-e432b44e-1638-480d-802f-0ff6e37630b4.png)
My installed version of CoppeliaSim is CoppeliaSim_Edu_V4_1_0_Ubuntu16_04, is this issue caused by some inappropriate configurations of the installed CoppeliaSim or something else? How do I solve this?
Repository: stepjam/RLBench
Issue: Draw a graph

Hi, I have two questions. 
1- how I can add a graph to my Coppeliasim environment for the gripper's joint position or joint velocity? Can I just use python for drawing gripper's graphs?

2- in rlbench tasks for example reach target, how does "target.get_pose()" detect the position of the target? by using the information from cameras and vision in the scene or the Cartesian coordinates of the target and other objects in the scene are sent to python by using API and something else?
Repository: stepjam/RLBench
Issue: Missing objects in block tasks?
Root cause: Validation Gap

A number of tasks seems to involve the robot interacting with an object that doesn't seem to exist in the scene (unsure at the moment if there and failing to render, or missing entirely). 

Examples can be seen in the official set of gifs:
Put item in drawer: https://drive.google.com/file/d/1EtsMAwrBlZfIEA2vC0ES8-myuiO2Ulz5/view
Reach and drag: https://drive.google.com/file/d/1N7lSujHf5ho_doKBqrCmbfqMGrVI1U5z/view
Slide block to target: https://drive.google.com/file/d/1YFgJKwD_Mj76RAlM2eMkByLl-ywJk-ED/view

It seems like it's some (but not all) tasks involving a block. In reach and drag it seems like a stick might also be missing, based on the task definition.

This also repros when I run the tasks on my machine as well (the first two anyway, haven't tested the third).

Update: it does show up in the Coppelia simulator, at least for reach and drag, just not in the rendered camera images.
Repository: stepjam/RLBench
Issue: Error about libsimExtROS2
Root cause: External Dependency Breakage

Hi, i'm currently trying to run the task_builder.py. My ROS, ROS2 are Noetic, Foxy on Ubuntu 20.04 with CoppeliaSim v4.5.1_rev4. running this in terminal get me this error 
(base) quyennt@quyennt:~/RLBench/tools$ python3 task_builder.py
python3: symbol lookup error: /home/quyennt/Downloads/CoppeliaSim_Edu_V4_5_1_rev4_Ubuntu20_04/libsimExtROS2.so: undefined symbol: _ZN15image_transport14ImageTransportC1ESt10shared_ptrIN6rclcpp4NodeEE
I tried to reinstall the ROS, both running terminal with/without conda but the error stays the same. Can anyone give me some help?
Thank you !

same error on my end - just a different symbol (`_ZTIN6rclcpp19QOSEventHandlerBaseE`)

Full error:
```
error: library ( /home/dev/CoppeliaSim/libsimExtROS2.so ) load: "Cannot load library /home/dev/CoppeliaSim/libsimExtROS2.so: (/home/dev/CoppeliaSim/libsimExtROS2.so: undefined symbol: _ZTIN6rclcpp19QOSEventHandlerBaseE)"
```
Specs:
```
OS Version: Ubuntu 22.04
CoppeliaSim: 4.5.1
PyReq: Mainline (for Coppelia 4.1, NOT 4.6.0-beta) 
ROS: ROS2 Humble
```
Please @ me if any solution is found.
Repository: stepjam/RLBench
Issue: How to read end-effector's velocity out of demos
Root cause: Documentation Deficiency


You can get the end-effector pose from the [observations](https://github.com/stepjam/RLBench/blob/master/rlbench/backend/observation.py) of the demos: e.g. `obs.gripper_pose`. You can then calculate the velocity based on the pose from time `t` and `t-1` and using the simulation timestep amount (which is `50ms` by default).

Happy to accept a pull request if you would like to add this in as a permanent observation in the 
observation.py file.

Best,
Stephen
Repository: stepjam/RLBench
Issue: BUG: assertion condition mismatching
Root cause: Validation Gap

At [gripper_action_modes.py line 61](https://github.com/stepjam/RLBench/blob/55e723d9400b54d37e6077ea2753e64874bc4639/rlbench/action_modes/gripper_action_modes.py#L61C34-L61C34), the compound inequality mismatches the assertion:
```
if 0.0 > action[0] > 1.0:  # which is equivalent to 0.0 > action[0] AND action[0] > 1.0
    raise InvalidActionError(
        'Gripper action expected to be within 0 and 1.')
```
which should be:
```
if 0.0 > action[0] or action[0] > 1.0:
    raise InvalidActionError(
        'Gripper action expected to be within 0 and 1.')
```
or
```
if not 1.0 > action[0] > 0.0:
    raise InvalidActionError(
        'Gripper action expected to be within 0 and 1.')
```
![image](https://github.com/stepjam/RLBench/assets/108717066/368175fd-2167-4383-a22b-f87aef980c6a)
Repository: stepjam/RLBench
Issue: Question: How to set the table texture (domain randomization)
Root cause: Documentation Deficiency

Hello. I've been trying to test the domain randomization module from `domain_randomization.py` and `domain_randomization_scene.py`, but so far I'm not able to point to the right images directory that contains the required textures. I'm trying to set the texture of the table the arm sits on to a random texture, so I was trying using the modules from `sim2real` to no avail. I've tried using the textures from the test folder, but it keeps crashing on me.

It'd be great if you could help me get the domain randomization module running, or even better, how to set the texture of the table (as this is a compound shape, I'm not sure how you set a texture in this case)

I checked the domain_randomization module and noticed how you handle compound shapes. I've got the table texture changing in my experiments :smile:. In case anyone need this as well, [here](https://github.com/stepjam/RLBench/blob/55e723d9400b54d37e6077ea2753e64874bc4639/rlbench/sim2real/domain_randomization_scene.py#L73) is the snippet I mentioned.

```python
    try:
        obj.set_texture(texture, **TEX_KWARGS)
    except RuntimeError:
        ungrouped = obj.ungroup()
        for o in ungrouped:
            o.set_texture(texture, **TEX_KWARGS)
        self.pyrep.group_objects(ungrouped)
```
Repository: stepjam/RLBench
Issue: randomization after reset_to_demo
Root cause: Algorithmic Error

Hi,

Thank you for your wonderful implementation. 

When I evaluate task 'open_fridge',  I notice that even after calling the [reset_to_demo](https://github.com/stepjam/RLBench/blob/master/rlbench/task_environment.py#L164) function, the initial object pose configurations sometimes (although not always) remain different compared to the data. In this situation, I observe that the [init_episode](https://github.com/stepjam/RLBench/blob/master/rlbench/task_environment.py#L78) function [attempts](https://github.com/stepjam/RLBench/blob/master/rlbench/backend/scene.py#L126) different numbers of times. I hypothesize that the sampling-based planner used in RLBench introduces a degree of randomization, which leads to different outcomes in the [task validation](https://github.com/stepjam/RLBench/blob/master/rlbench/backend/scene.py#L134) process. Have you made similar observations? If so, do you have any suggestions on how this issue might be resolved?


I have the same problem. Do you have a solution now?

marked as same problem

@TongZhangTHU have you solved it?
Repository: stepjam/RLBench
Issue: tools/dataset_generator.py
Root cause: Documentation Deficiency

Hi，I want to produce some RLBench data for my current research, I run this file to only get some png images, why no action data?I think most imitation learning tasks require groundtruth actions, can you supplement the preservation of action?

actions depend on arm_action_mode and I found out the actions from arm mode JointVelocity and JointPosition can be found in observations. For other arm modes, you should find out how these actions are generated and I think most of them can be calculated with the observations from the environment. 

> actions depend on arm_action_mode and I found out the actions from arm mode JointVelocity and JointPosition can be found in observations. For other arm modes, you should find out how these actions are generated and I think most of them can be calculated with the observations from the environment.

Thanks for your reply. By reading the source code, I think I should understand the meaning of the action. Here's a question for you: Does RLBench give you a range of data? Because I want to use the maximum and minimum values of the data to do some discretization.
Repository: stepjam/RLBench
Issue: [Feature Request] Can the gym environment provide depth images and point clouds?
Root cause: Unimplemented Feature Gap

https://github.com/stepjam/RLBench/blob/46cb9d97003961428f491dd7cc75433f6ab351dd/rlbench/gym/rlbench_env.py#L50-L63

Although the original RLBench environment includes depth images and point clouds in the observation, the gym version does not provide such information. They are important if researchers want to render images from new viewpoints (e.g., [Robotic View Transformer](https://robotic-view-transformer.github.io/)). Could you please add them in the code?

I can modify the code and pull a request if you like it.

Kind regards,
Zhuoheng
Repository: stepjam/RLBench
Issue: ImportError: No module named 'rlbench.sim2real'
Root cause: Environment Incompatibility

`>>> import rlbench
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File /vrep/lib/python3.5/site-packages/rlbench-1.0-py3.5.egg/rlbench/__init__.py", line 5, in <module>
    from rlbench.sim2real.domain_randomization import RandomizeEvery
ImportError: No module named 'rlbench.sim2real'
`
Hi there, after I install rlbench, the following error occurs. Thanks.

I can successfully run PyRep examples like : example_baxter_pick_and_pass.py

After commenting our the sim2real lines in __init__.py for rlbench, the problem goes away.


Thanks for reporting! Yes this was a bug. If you re run the installation of RLBench, this should now work.
Repository: stepjam/RLBench
Issue: To be compatible with openai gym environment
Root cause: Unimplemented Feature Gap

Hi, stepjam!
     I really appreciate what you've done and this is a good benchmark for researchers in robotics. But I hope if you can make the tasks be compatible with OpenAI's gym environment since there are a lot of RL algorithms developed by others which we can use as the baseline performance easily. Thanks.
   

Hi @KuKuXia ,
The tasks have the same methods that Gym Envs have, so what would be the additional benefit of this? The reason why I haven't already done this is that because RLBench is unique in the sense that it has the ability to get demos, so it's not super clear if that would add necessary fluff to the code. I am open to pull requests though :)

Hi, stepjam!
Thanks for your replay. I've changed my project to use the IKEA Furniture Assembly Environment. 
Since it's compatible with openai's gym environment and I can mainly focus on the RL algorithms. I'm using openai's baseline as the baseline performance quickly without additional code modification. I hope someone can make a whole working code using this environment and make it public now.




I've had quite a few emails about gym support, so I perhaps I should add in. Expect to see this live some time tomorrow. 

Hi, stepjam,
Thanks for your great work. I'm looking forward to seeing your new features now. I'll keep watching your repo and wish it will help build my project.

Now merged to main branch. Enjoy!
Repository: stepjam/RLBench
Issue: Saving ttm issue
Root cause: Validation Gap

Hi,stepjam,I have an issue of saving the simulation ttm file, the prompt of task builder thrown error as following: 
pyrep.errors.ObjectIsNotModelError: Object '%s' is not a model. Use 'set_model(True)' to convert.
QObject::~QObject: Timers cannot be stopped from another thread
QMutex: destroying locked mutex
can you give me some advice of solving it?
best, regards.
Repository: stepjam/RLBench
Issue: utils.get_stored_demos and dataset_generator.py don't agree on mask image format, results in corrupted handles for stored demos
Root cause: Interface Contract Mismatch

I've been using the dataset_generator.py script to generate demos for RLBench tasks, which is great.

However, when I try to load them, I find that the segmentation mask handles are different when reading from a stored dataset (i.e. by calling `utils.get_stored_demos`) compared to loading them online.

I've tracked this down to the fact that the dataset_generator.py script converts 3-channel segmentation masks into the [0, 255] range and converted into an int before storing as pngs: https://github.com/stepjam/RLBench/blob/master/tools/dataset_generator.py#L96

However, when the demos are read in `utils.get_stored_demos`, the PNGs generated by this process are read here: https://github.com/stepjam/RLBench/blob/master/rlbench/utils.py#L285, which converts them from 3-channel masks to 1-channel masks using this function: https://github.com/stepjam/RLBench/blob/master/rlbench/backend/utils.py#L223

As you can see, the mask is multiplied by 255 again, and converted into int. However, when it is read in, it is already on the range [0, 255], and of type uint8. Therefore multiplying by 255 causes some overflow, and i can't think of a way to invert the resulting overflow.

I've currently modified rgb_handles_to_mask to remove this conversion for my purposes, but im not sure if there are other use cases where that conversion should be handled. Is `dataset_generator.py` the only producer of PNGs which would be consumed this way?
Repository: stepjam/RLBench
Issue: How to edit obj part textures  

Hi, thanks for your splendid work.

I would like to modify the specific obj's part texture, such as the buttons/handle on the oven. 
From [this line](https://github.com/stepjam/PyRep/blob/076ca15c57f2495a4194da03565891ab1aaa317e/pyrep/objects/shape.py#L408) in pyrep, it seems to be possible to modify by code line, not modifying the obj texture itself. Is there any guidance or example code of this?