zhiminy/EvalEng huggingface dataset:
Repository: stepjam/RLBench
Issue: Dataset Generator file not working
Root cause: External Dependency Breakage

Hi Stephen, first of all thank you so much for building this amazing tool.
I am trying to use your dataset_generator.py file but I am getting this error

`Traceback (most recent call last):
  File "/usr/lib/python3.5/multiprocessing/process.py", line 249, in _bootstrap
    self.run()
  File "/usr/lib/python3.5/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "dataset_generator.py", line 148, in run
    if my_variation_count >= task_env.variation_count():
AttributeError: 'TaskEnvironment' object has no attribute 'variation_count'
Data collection done!
`
Any guidance on this would be hugely appreciated!
Thanks so much,
Sambhav

Can confirm, but with a different error.

I ran:
` python3 dataset_generator.py  --episodes_per_task 1 --processes 1`

It errorred out with 

```
Initialization successful.
Process 0 collecting task: take_money_out_safe // variation: 0
Process Process-2:
Traceback (most recent call last):
  File "venv/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "venv/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "dataset_generator.py", line 163, in run
    task_env = rlbench_env.get_task(t)
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/environment.py", line 166, in get_task
    self._prev_task.unload()
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/backend/task.py", line 283, in unload
    self.cleanup()
  File "/home/user/.local/lib/python3.7/site-packages/rlbench/tasks/take_money_out_safe.py", line 110, in cleanup
    for m in self.money_list:
AttributeError: 'TakeMoneyOutSafe' object has no attribute 'money_list'
Data collection done!
```

I suspect this is due to a version mismatch in multiprocessing, that for some reason `_prev_task` is being assigned a task before the actual instantiation of the task, triggering unload + cleanup of a task that hasn't even been initialised. Could this be the case? I'm using Python 3.7.6.

Please advise. Thanks a ton.

Hi Both,

@sambhavaggarwal . Please make sure you have the latest version of RLBench **installed**. It looks like you might have just pulled the new dataset_generator without re-installing RLBench.

@nevakanezzar . Thanks for pointing out! Expect a fix very soon.

@nevakanezzar . Please reopen if this does not resolve the issue. 

Seems to be fixed! Thanks, this is great. I see some of the tasks failed to run, I'll post their error messages in a new issue.
Repository: stepjam/RLBench
Issue: please add examples
Root cause: Documentation Deficiency

Would it be possible to add a full working example with tf2.0 keras? The current examples seem to only have numpy random actions

Hi. The examples are meant to be as lightweight as possible. I feel like having the examples littered with machine learning framework code would be a distraction to the core RLBench things I'm trying to show.
Nonetheless, I'm happy to take pull requests for tf2.0 examples in a new '_tf_examples_' folder if anyone is interested :)
Repository: stepjam/RLBench
Issue: Manually setting poses of objects in scene to recreate previously used scenes
Root cause: Unimplemented Feature Gap

There is probably a simple way to do this but I've been digging through source for a while and don't yet see it. Instead of randomly initializing a scene, I would like to place the objects in the scene at set poses in the workspace. My use case is I want to record demos where objects in the scene are placed randomly in the workspace, and then later I want to be able to recreate those same scene configurations. Is there a simple way to do this with the existing infrastructure?

Hi Adam,
Not in the existing infrastructure, but it's easy to add in. However, I was planning to make 'reset to demo' a bit more general, where you can specify any point along the demonstration to reset to; this is a little more tricky.
Anyway, I can roll this in to V1.0.5, which I hope to push out in the next day or two.

Hi Stephen,
That would be really helpful to me. Right now I just need to take whatever initial configuration was set for a previously recorded demo, and make that the current configuration.
Thanks!
Repository: stepjam/RLBench
Issue: Why does segmentation mask have 3 channels?

The segmentation masks all have 3 channels, but only the first channel contains meaningful numbers and the other 2 channels are all zeros. Is there a reason behind this?

Hi Adam,
This is how CoppeliaSim returns segmentation masks. It first fills the R channel (until 255), then G, then B. I think they do it this way to make it easier to visualise the masks; however, maybe for RLBench we should convert it to a one channel standard. Would you prefer that?

Hi Stephen, one argument for making it single channel is it cuts down on storage space. I'm storing all my data in HDF5 files and loading the files can take longer the larger the file size. Also I spent a while thinking about why there would be 3 channels until I realized only one channel actually had data in it, so maybe from an interpretability standpoint it would also be helpful.
Repository: stepjam/RLBench
Issue: [Feature Request / Help] rgb_array rendering for gym
Root cause: Unimplemented Feature Gap

New to the library — right now, gym.render only supports 'human', so it opens a window for the camera. Would it be possible to get the rgb_array for the scope camera just like for the wrist/shoulders, and return that from env.render("rgb_array", width, height)? 

Could set up a camera with task builder and invoking that camera 

Sure, I'll add this in to V1.0.6.
Repository: stepjam/RLBench
Issue: Gym info should be dict
Root cause: Interface Contract Mismatch

In RLBench_env.py, step() function should return {} instead of None

Actually if we really want to complete the Gym env properly, it should probably be a GoalEnv (since it's mostly sparse reward tasks with a single goal), and we can include is_success instead of {}. The compute_reward() function will be useful for people using replay buffer tricks.
Repository: stepjam/RLBench
Issue: Imitation learning prediction - Question
Root cause: Documentation Deficiency

For the imitation learning example script, is it possible to send the predicted actions to the scene instead of just computing the loss?

By this I mean, can I test the robot with my networks' prediction?

I basically know how to do this with PyRep, but not with RLBench. Would really appreciate it if you can advise.

Thanks for the great work and effort you are putting on this!

Something like this?
https://github.com/stepjam/RLBench/blob/master/examples/single_task_rl.py

Yes, very much appreciated :)
Repository: stepjam/RLBench
Issue: Gym doesn't support End Effector-related action mode (EE_POSE)
Root cause: Documentation Deficiency

Setting action_mode = ActionMode(ArmActionMode.DELTA_EE_POSE) for the gym reach_target task, for example, gives rlbench.task_environment.InvalidActionError: Could not find a path. This is due to the action_space giving out of bounds possibilities. 

Edit: after looking into this it seems that the IK breaks even with tiny epsilon (< 0.0005) movement, perhaps when colliding into an object? not sure.

Yes the deltas need to be small because its using the Jacobians. If you want a larger delta, then you can use ABS_EE_POSE_PLAN or DELTA_EE_POSE_PLAN in V 1.0.6: https://github.com/stepjam/RLBench/blob/1.0.6/rlbench/action_modes.py#L45.

This will be merged into main soon. It uses path planning to get to a target rather than getting the joint angles via the Jacobian.

Please reopen if you have further questions :)

Thank you stephen! 

I can't reopen this issue — but could you tell me what delta for example the panda arm can be set to? I'm still getting errors w/ actions on the order of 0.01
Repository: stepjam/RLBench
Issue: Why does the arm move sometimes slowly somtimes fast?
Root cause: Configuration Error

Hi, I am using jaco for reinforcement in RLBench. I tried the RLBench/examples/single_task_rl.py and I have two questions:

1. How to set jaco arm correctly? 
At first, I just set like this:
`env = Environment(action_mode=action_mode, robot_configuration='jaco', obs_config=obs_config, headless=False)`
But it comes out an error:
```
Traceback (most recent call last):
  File "rlbench_test.py", line 44, in <module>
    obs, reward, terminate = task.step(action)
  File "/home/gck/.local/lib/python3.6/site-packages/rlbench/task_environment.py", line 147, in step
    (len(self._robot.arm.joints),))
  File "/home/gck/.local/lib/python3.6/site-packages/rlbench/task_environment.py", line 109, in _assert_action_space
    str(expected_shape), str(np.shape(action))))
RuntimeError: Expected the action shape to be: (6,), but was shape: (7,)
QObject::~QObject: Timers cannot be stopped from another thread
QMutex: destroying locked mutex
```
I think this is because the action number is not set correctly, so I go to your code (action_modes.py) and find how to set it. Then I add two lines and it runs normal:
```
action_mode.arm.action_size = 6
action_mode.action_size = action_mode.arm.action_size + action_mode.gripper.action_size
```
I don't know if this is a proper way, so I want to ask you for suggestion.

2. Why does the arm move sometimes slowly somtimes fast?
When I run the RLBench/examples/single_task_rl.py, I find the time of every loop is quite difference:
```
for i in range(training_steps):
    if i % episode_length == 0:
        print('Reset Episode')
        descriptions, obs = task.reset()
        print(descriptions)
    action = agent.act(obs)
    print(action)
    obs, reward, terminate = task.step(action)
    print('steps:',i)
```
In the simulation, one 'step' function sometimes lasts for 1s or 2s sometimes only for 1ms. It looks like [this](https://cloud.tsinghua.edu.cn/f/8e8890a49bc748eaa3e8/). You can see that it seems like sometimes the step function just stucks.

I am looking into this
Repository: stepjam/RLBench
Issue: dataset_generator Demos Image Loading
Root cause: Documentation Deficiency

Hello, 

I downloaded lots of demos for which I needed the images also loaded in the `Observation` object when I do `task.get_demos(n,live_demos=False)`. I noticed that in the code you have kept a bool on `image_paths`. Is there going to be support in the future for loading those images to memory ?   

I found the solution in the same method :) . You have built a great library !.
Repository: stepjam/RLBench
Issue: Stepping speeds
Root cause: Documentation Deficiency

What stepping speeds are you seeing for the base (i.e., delta joint velocity), no cameras? I'm seeing 0.008s flat, but this seems to be too slow for our research. For example, mujocopy sees 0.0024s per step.

I get ~0.0055s per step. You cant really compare against mujoco because it is comparing apples and oranges. Mujoco is a physics engine whereas CoppeliaSim (which is what RLBench is built around) is a robotics toolkit/engine which incorporates physics engines. So you get a lot more from CoppeliaSim than you would mujoco, and so step times are naturally going to be longer. 
By the way, 3 times slower isn't particularly that bad anyway, you could just spawn more environments to increase data throughput.
Best,
Stephen

Sounds good Stephen, thanks for testing!
Repository: stepjam/RLBench
Issue: Customizing environments programmatically
Root cause: Environment Incompatibility

What's the supported way to edit environments (the task ttms) programmatically? 

Could you elaborate?
You can interact with CoppeliaSim through PyRep, but that interface was intentionally hidden for the purpose of the benchmark. Could you give me an example of what you would want to do?
Best,
Stephen

Sure. Let's say I want to identify the position of a block object in the environment and move it upwards by some amount, but do it programmatically. (I feel this would encompass other common use cases, like let's say I want to put a vision sensor in a specific location.)

And just to expand on this use case: would you want demonstrations? (i.e. build your task using the task_builder tool?)

If not, then you would be better off using [PyRep](https://github.com/stepjam/PyRep)

Yeah, demonstrations would probably necessitate the use of a GUI anyways, so, I guess I'd have to use PyRep. Would it be possible for you to demonstrate a barebones script such as, i.e., loading the RLbench reach_target task and using pyrep to moving the boundary box upwards, or something simple like that? It would be helpful, thanks :) 

PyRep already gives an example of how you can build a simple reaching environment. See here: https://github.com/stepjam/PyRep/blob/master/examples/example_panda_reach_target.py

I think one important thing to bear in mind is not to confuse what RLBench is. It is a standard set of tasks, where people can add new tasks using the **standard scene setup and standard sensors** . Also people can easily automatically generate demonstrations when building their tasks. 
So if you want to start moving/adding sensors, changing the scene, changing the robot, etc, then this should be an indication that RLBench is not what you want, as this moves a way from the purpose of the benchmark.

PyRep on the other (which is what RLBench uses under the hood) is more general and allows modification of the CoppeliaSim scenes. So you can just build you scene using the CoppeliaSim GUI (you are welcome to even use the RLBench ttm as a starting point: https://github.com/stepjam/RLBench/blob/master/rlbench/task_design.ttt) and then you can just add a little programming glue to build your RL env (like in the example I shared in the previous comment)

Hope that help

I want to use task-builder.py to customize the task, but my CoppeliaSim software seems to have incomplete functionality.
![83a0233d2177ec66c18c5056eb37f8f](https://github.com/user-attachments/assets/e6998dfb-e222-4472-8270-67205334444c)
![Uploading 7e001ffb14a0bf611fd70bd0e4429f4.png…]()

As shown in the above figures, I am unable to save the scene and model under the File button; I cannot find options such as dummy and sensor under the add button. But I saw online that other people's CoppeliaSim has these features.
 
By the way, what should I do if I want to make modifications based on some tasks in rlbench?
Repository: stepjam/RLBench
Issue: Success returns the same value for both reward and done
Root cause: Documentation Deficiency

https://github.com/stepjam/RLBench/blob/8ff9507a7b6be5b374f6b7e619b4b7eda1f9fd57/rlbench/backend/task.py#L261

Am I missing something?

sparse reward, so rew = 1 = done = True, rew = 0 = done = False 

Hi both. What @bycn is not quite right; what he said is true on the environment level (calling `env.step`). However, that particular success function was originally going to return both success value and if any constraints were breached, but I decided not to include it, hence why it returns the same value for both. I will refactor this out in future. 

I wouldn't worry too much about it as its a backend function and you shouldn't need to use it.
Repository: stepjam/RLBench
Issue: swap arm in imitation learning
Root cause: Documentation Deficiency

I'm trying to to change the arm to Mico in imitation learning script. But I got some problems as:

 
```
File "/home/ggb/.local/lib/python3.6/site-packages/pyrep/robots/end_effectors/gripper.py", line 146, in get_touch_sensor_forces
    raise NotImplementedError('No touch sensors for this robot!')
    NotImplementedError: No touch sensors for this robot!
```

I tried mico,jaco,swayer in several tasks, all they went wrong with no touch sensor error. So I set the obs_config/gripper_touch_forces to False and try ReachTarget task since I think this task may not need touch sensors. But it come to the second problem.




```
 Warning: SBL: Skipping invalid goal state (invalid bounds)
         at line 310 in /home/marc/Documents/ompl-1.4.2-Source/src/ompl/base/src/Planner.cpp
Warning: SBL: Skipping invalid goal state (invalid bounds)
         at line 310 in /home/marc/Documents/ompl-1.4.2-Source/src/ompl/base/src/Planner.cpp
```

the arm has no action and the scene keeps changing. I think it may because the get_demos() function cannot get proper planning.



So can you give me some advice for how I can adjust task to different arms  in imitation learning to get right demos.




As you pointed out, not all the arms have touch sensors. This is why the example sets that observation to false: https://github.com/stepjam/RLBench/blob/master/examples/swap_arm.py#L21

Regarding your second point: this is expected behaviour. The Mico has a very short workspace area due to the arm link lengths, and so many of the task configurations will not be valid for that arm. With the reacher, the mico can do some of the configurations (it will keep resetting the task until a valid configuration is found). However, when you swap this out to a more complex task (i.e. the open oven task) then this will most likely throw an exception because that task is impossible to do with the Mico.

Hope that helps :)
Repository: stepjam/RLBench
Issue: slide_block_to_target rendered incorrectly in camera views
Root cause: Algorithmic Error

For **slide_block_to_target** task, the object and target are not rendered correctly in cameras (see attached screenshots. I haven't seen this issue in any other task.

![Screenshot_2020-04-15_17-32-09](https://user-images.githubusercontent.com/1252912/79402308-4de96f80-7f40-11ea-949b-c576efa81a95.png)
![67](https://user-images.githubusercontent.com/1252912/79402335-66f22080-7f40-11ea-9813-175b947be8f2.png)


This is now resolved in branch 1.0.7, which will be merged to main soon :)
Thanks for reporting.
Repository: stepjam/RLBench
Issue: How did you record those nice videos?

This is just a question, not an issue.

Could you please share how you recorded your videos? I used the video recorder in V-REP, but the videos contain all the red or blue lines in simulation. Did you add a camera (with high resolution) and make it rotate programmtically? If so, could you please share that code?

Yes I did add a high resolution camera and made it rotate. I'll push the script to the repo shortly :).

Great, thank you! Looking forward to it.

Has this been added yet? I am interested in getting higher resolution videos and am having a hard time figuring out these cameras in CoppeliaSim.

Sorry @ascane , this got buried in my todo list. @adamconkey: I'll get this added in today.

Awesome, thanks!
Repository: stepjam/RLBench
Issue: Questions on the base frame of Panda robot
Root cause: Documentation Deficiency

Hi, I'm enjoying your awesome implementation research API for deep RL and its friends.

Currently, I'm varying the action space of the Panda robot, whilst, I'm struggling with 

engineering the action space of PandaRobot to delta EE or abs EE that require

defining the tip orientation as a quaternion.

What I want to do is fully exploit the cartesian action space (6DOF),

so what I'm trying to do is the first map the orientation part of policy output to axis-angle representation,

confining the z-axis of the normalized vector to be negative, which in my opinion makes the EE of 

PandaRobot head downward (toward the table). After that, I acquire the confined axis-angle SO(3) 

to a unit quaternion, to solve IK via PyRep,

However,  what I have found is that the axis-angle representation of EE('Panda tip') of Panda w.r.t. 

its base (what I believe 'Panda' in CoppeliaSim scene hierarchy), as shown in the picture, is not 

heading toward the table. 

I noticed that the z-axis of the base frame of Panda (as shown in the picture of wire-framed Panda)  is 

not heading upward.

I don' t know if Coppelia sim use the PoE formulation for kinematics, but is there a reason for this 

definition of the base frame?

Thanks in advance!

Best regards,

Cheol-Hui.

![SeemsWrong](https://user-images.githubusercontent.com/20310517/79567940-030e5a00-80f0-11ea-9c59-69a9b2c1f7ba.png)


Hi Cheol-Hui,
All of the ee_pose action spaces operate in the panda_tip **w.r.t world** not the arm base. 
Sorry, I should make this more clear in action_modes.py. I'll add some comments in version 1.0.7.
Best,
Stephen

@mch5048 I was thinking about expanding the end-effector action space so that you could operator in world space and gripper space (currently it is world space). So instead of:
ABS_EE_VELOCITY, DELTA_EE_VELOCITY, ABS_EE_POSE, etc
we would have:
ABS_EE_VELOCITY_WORLD_FRAME, DELTA_EE_VELOCITY_WORLD_FRAME, ABS_EE_POSE_WORLD_FRAME, ABS_EE_VELOCITY_EE_FRAME, DELTA_EE_VELOCITY_EE_FRAME, ABS_EE_POSE_EE_FRAME, etc

Does that sound useful?
Repository: stepjam/RLBench
Issue: dataset generator not working for new task
Root cause: Validation Gap

I have created a new task and after variation validator, I tried to get the demos of that task. But running the dataset said that it couldn't find my task.


cmd:python3 tools/dataset_generator.py --tasks left_target --episodes_per_task 2
out:Traceback (most recent call last):
  File "tools/dataset_generator.py", line 295, in <module>
    app.run(main)
  File "/home/vicky/.local/lib/python3.6/site-packages/absl/app.py", line 299, in run
    _run_main(main, args)
  File "/home/vicky/.local/lib/python3.6/site-packages/absl/app.py", line 250, in _run_main
    sys.exit(main(argv))
  File "tools/dataset_generator.py", line 265, in main
    raise ValueError('Task %s not recognised!.' % t)
ValueError: Task left_target not recognised!.


Was Python Interpreter Issue.
Repository: stepjam/RLBench
Issue: Gifs are buggy?
Root cause: Documentation Deficiency

https://drive.google.com/drive/folders/1TqbulbbCEqVBd6SBHatphFlUK2JQLkYu

Ex., the block to target tasks look like they are grabbing nothing to the goal

never mind, the cameras just don't see the block. Weird, since i'd assume this should be necessary for the task to be completed.

Hi,
No you were originally right... the block was previously not rendered, but this was fixed a while ago. The gifs were made before the bug fix.

Would it be possible to link the commits for the fix? 

Found it #59