zhiminy/EvalEng huggingface dataset:
Repository: stepjam/RLBench
Issue: Add option for additional timesteps at beginning/end of recorded demo
Root cause: Documentation Deficiency

I have an issue with my recorded demos that they seem to stop recording prematurely even though technically the success conditions for the task have been met. For example, it has satisfied the proximity sensor success condition but the robot is still in motion, and I would like at the very end of the demo to be near zero joint velocity and in a static position.

I tried adding a success condition for having zero velocity at a waypoint, but I couldn't get it to work for whatever reason. My new solution is to hack the `get_demo` function of `Scene` by adding this to the beginning and end of the demo:
```        
for _ in range(10):
    self.step()
    self._demo_record_step(demo, record, func)
```
I am wondering if you think it'd be good to make this a feature of the API such that you can parameterize my hack and allow the user to specify how many additional timesteps they want recorded at the start or beginning.

Alternatively, is there a better way to set the success condition for the robot being in a static configuration at a waypoint?


Hi Adam,
Have you used ConditionSet? see [here](https://github.com/stepjam/RLBench/blob/master/rlbench/backend/conditions.py#L146)

[Here is an example of it being used](https://github.com/stepjam/RLBench/blob/8ff9507a7b6be5b374f6b7e619b4b7eda1f9fd57/rlbench/tasks/take_lid_off_saucepan.py#L15). In this case, it's checking that the lid is in the prox sensor and that it is still grasped.

Best,
Stephen

> Alternatively, is there a better way to set the success condition for the robot being in a static configuration at a waypoint?

The best way, as you suggested, is to have a new success condition for the arm being static. I think this would be a useful success condition to rlbench, so please make a PR :)

Hm I didn't know about the ConditionSet, I was under the impression that by registering a list of conditions they would be AND-ed together. I will try the condition set with a static condition and if it works I can submit a PR for the static condition.

Yes fair point, I agree, they should be AND-ed. Happy to accept a PR for this, otherwise I'll add it on my TODO list to eventually get to.
Repository: stepjam/RLBench
Issue: How to serialize FFI CData to disk to restore state?
Root cause: Documentation Deficiency

I am hoping to use the `get_state` and `restore_state` functions on the tasks to re-initialize an environment exactly as it was in a recorded demo. This does work, but my problem is I want to save the state to disk (I don't care what format) and be able to reload it later to restore the state. It looks something like `(<cdata 'char *' 0x55c54298da10>, 5)`, and it's this cdata type that is causing trouble no matter how I try to (de)serialize it. I tried pickle, dill, writing and loading just as binary. I have been trying for hours and am quite stuck. Do you know of a way to handle this? 

https://cffi.readthedocs.io/en/latest/ref.html#ffi-cdata-ffi-ctype
You need to call ffi.string(cdata) and then just write the bytes as binary

I had tried that, writing to bytes I could do, it's loading them back into cdata that `restore_state` can use that was the challenge and I had no success with that part. I tried various things with reading the bytes into buffers but just couldn't get it to work.

Hi,
This functionality already exists:
```
task = env.get_task(ReachTarget)
demos = task.get_demos(2, live_demos=live_demos)
task.reset_to_demo(demos[0])
# The task will now be the same starting config as the demo
```

Thanks Stephen, I think this works for me.

@adamconkey did you ever get this to work or did you just use stephen's "solution"? I'm trying to do multiprocessing and the cdata has to be serialized (i.e., i have to send the bytes instead), but loading it back in gives some segmentation fault errors.
Repository: stepjam/RLBench
Issue: Success sensor with block?
Root cause: Documentation Deficiency

I have a block and a success sensor. It registers a sucess when the block.get_position() - sensor.get_position() is nonzero, and not only this, but when it is closer it does not register success (so it can't be an epsilon ball). When viewing images, it seems like it takes one of the corners of the block as the position. What's the right way to do this so it senses the middle of the block?

Hi,
Maybe I am misunderstanding, but why not just make the proximity sensor smaller?

Basically the question boils down to how block.get_position() / sensor.get_position() are calculated. I expect it to be the centre of the object but not sure if this is true from visualization

Yes, calling get_position on a pyrep object will give you the position of the objects centre.
When you say you have a 'success sensor', what do you mean? Do you mean a [proximity sensor](https://github.com/stepjam/PyRep/blob/master/pyrep/objects/proximity_sensor.py) or [condition](https://github.com/stepjam/RLBench/blob/master/rlbench/backend/conditions.py)?

Proximity sensor. It seems not to use the position() to calculate success

No it doesn't use get_position. You can if you want define your own Condition that calculates an explicit distance within some delta (using get_position), however I would suggest using the [proximity condition](https://github.com/stepjam/RLBench/blob/master/rlbench/backend/conditions.py#L43) as this is much nicer to visualise when defining your task. You can make your proximity sensor as small or as big as you like to get the same effect as a get_position approach would give you :)

Please reopen if you still need help with this :)
Repository: stepjam/RLBench
Issue: dataset generator results?
Root cause: Documentation Deficiency

https://github.com/stepjam/RLBench/blob/8ff9507a7b6be5b374f6b7e619b4b7eda1f9fd57/tools/dataset_generator.py

Has this been run on the current tasks and uploaded somewhere? 


Hi,
No, the plan is to eventually do this, but because the benchmark is still in its 'teething' phase, I figured there would not be much point as there are still changes happening that may invalidate the generated data. Also, the amount of demos people would want would vary; i.e. for RL it way only be 10, but for imitation learning it may be thousands. So for now, I think it's best that people run the script themselves for the tasks they want.
Repository: stepjam/RLBench
Issue: rlbench_gym example import bug
Root cause: Interface Contract Mismatch

When I use "python examples/rlbench_gym.py", 
I meet
![Screenshot from 2020-05-28 12-07-39](https://user-images.githubusercontent.com/33066765/83098066-32938780-a0dc-11ea-9235-b29e63cb97c7.png)

I solve this problem by changing the order of import in examples/rlbench_gym.py, where I first import rlbench.gym and then import gym.

I'm not sure what happened. Just for reference.
I use python3.7.
Repository: stepjam/RLBench
Issue: Using segmentation mask

Hi,

I'm trying to create a camera in the scene and then get the segmentation mask from it. How would I do this? 

Context for my specific use case: This is after env.launch() is called, I do      
`   self.cam = VisionSensor.create([img_size, img_size])
`
and it works, but how do I get the mask from this? For the cameras given in the scene there exists corresponding mask cams,  i.e., for cameraovershoulderleft there is cameraovershouldlerft_mask, but i check the ttm and there isn't any cameraovershoulder_mask in the scene.

looking at this commit https://github.com/stepjam/RLBench/commit/d0006faeb195b52a6c19306222c05d7cc40f35b9
maybe the mask cams are done manually? How can I dynamically make a cam have a corresponding mask one?

Found it

> Found it
Thanks for your great comments . Can you tell me how to solve this problem?
Repository: stepjam/RLBench
Issue: May I ask why the robot arm can only move a little when I run single_task_rl.py
Root cause: Documentation Deficiency

First of all thank you for building this great project.
when I run single_task_rl.py the robot arm can only move a little .
![recording_2020_06_03-01_20-23](https://user-images.githubusercontent.com/55341540/83550372-55d59100-a539-11ea-8cf3-5aff634b92d3.gif)
What code do I need to modify to make the robot arm [move like this.](https://cloud.tsinghua.edu.cn/f/8e8890a49bc748eaa3e8/)
Looking forward to your reply.


In that example, the action mode is joint velocities, and the actions are sampled from a normal. This makes the robot end-effector stay somewhat in the same location. You could multiply this action by some constant to increase the random velocities.
Repository: stepjam/RLBench
Issue: Fixed pose for demos? 
Root cause: Documentation Deficiency

Hi,

For my use case I would like the demos to be able to work with a fixed pose (i.e., gripper remains upright the entire time), how would I go about doing this?  Thank you!

As an aside, could you possibly explain what "gripper touch forces" are? My understanding is they are a measure of the force and not part of the input.. But my main question is, how can I recreate an end effector action based on the demo? I guess as soon as the delta in gripper.get_open_amount() after a step is positive, we start a gripper open/close action correspondingly?


Hi,
For the first point, I don't see how this will be possible for the majority of the tasks as they will not be solvable if you restrict the rotation of the gripper, and so it would not be possible to even get demos. 
For the second point; yes the gripper forces are an optional observation you can request. The only way to control the end-effector currently is the last value in the action vector (which should be between 0 and 1). If < 0.5 then it will close until it cannot close anymore, and if > 0.5 it will try and fully open.
Best,
Stephen 

Hi James,
Thanks for the response. 
For the first point, let's say I create a custom task that is solvable with the pose fixed. How would I go about getting demos with fixed pose?
Repository: stepjam/RLBench
Issue: Segmentation Mask Labels
Root cause: Documentation Deficiency

Is there a way to recover the actual name of the class from the segmentation mask? In the case that an object is occluded completely in the scene, there's no way to identify the labels anymore. 

You can get the CoppeliaSim object handles. With that, you can access the object if you wish, and then consequently access the 'name' of that object. For example: 
```python
# Here we tell rlbench to return the mask on the form of object handles 
# (rather than a tytpical segmentation mask)
observation_config.left_shoulder_cam.masks_as_one_channel = False
# ... now give the config to rlbench ...
# ... get observation ...
object_handle = left_shoulder_mask.mymask[0, 0]  # Get top right handle

from pyrep.objects.object import Object
object_name = Object.get_object_name(object_handle)
```
Repository: stepjam/RLBench
Issue: quesionts when run the rlbench.gym
Root cause: Documentation Deficiency

Hi, thanks for your project.
I have some questions when using rlbench.gym.

1. The windows always is created and shows the observatioins when env.step() **although I have annotation the env.render()**.
Could you please tell me _**how to stop creating and showing**_ when I using rlbench.gym? I just want to use it in headless mode.
2. Also, I'm using the rlkit to test the rlbench, which **_recommands two envs for exploration and evaluation_**.
But it seems that the rlbench.gym **_cannot launch them in one terminal at the same time_**, could you help me ?
3. Another question is that I **cannot save the logs when running** the rlbench.gym.
The error is :
![image](https://user-images.githubusercontent.com/29306735/85355179-9e7fd900-b53e-11ea-9d15-31db222a535f.png)
I guess the reason is the main process is rlbench.gym, then I launch **_another process_** to save the parameters.
Could you please give me some advice?

Thanks and looking forward your replay.

Hi

1. It sounds like you are using 'human' rendering. If you want headless, then you instead need to call render with mode 'rgb_array', as you would with a standard gym env.

2. I don't know about rlkit specifically, but you need to make sure that `gym.make` (which in turn launches a RLBench/CoppeliaSim instance) is called inside each new process. I imagine rlkit is not doing this.

3. What are you trying to log? If is the parameters of the network then thats a question for rlkit rather than rlbench.

All the best,
Stephen

> Hi
> 
> 1. It sounds like you are using 'human' rendering. If you want headless, then you instead need to call render with mode 'rgb_array', as you would with a standard gym env.
> 2. I don't know about rlkit specifically, but you need to make sure that `gym.make` (which in turn launches a RLBench/CoppeliaSim instance) is called inside each new process. I imagine rlkit is not doing this.
> 3. What are you trying to log? If is the parameters of the network then thats a question for rlkit rather than rlbench.
> 
> All the best,
> Stephen

Thanks, the 3rd question is that the rlkit need to log the parameters of env, and I have solved annotated it and works well.
As for the 2nd question, I just need to launch one env and use this one.
Thanks for your advice!
Repository: stepjam/RLBench
Issue: gripper action causes very slow speeds
Root cause: Algorithmic Error

The step function in task_environment.py is very slow if there is a gripper action (about 5x - 10x slower). Is there a reason for this while loop?
```
while not done:  
    done = self._robot.gripper.actuate(ee_action, velocity=0.2)  
    self._pyrep.step()  
    self._task.step()  
```
If I change it to execute the gripper.actuate function only once, it runs much faster.

And why is no arm action executed if there is a gripper action? Can't they be executed at the same time?

I have seen that my last point has already been fixed in 1.0.7. I was still running 1.0.6.

However, I still don't understand why the gripper action has to be run in a loop.

The gripper action has to be run in a loop because the gripper closes at a certain velocity; meaning that we have to step the simulation several times in order to close the gripper fully.
Repository: stepjam/RLBench
Issue: How can I control the task sample?
Root cause: Documentation Deficiency

In the imitation_learning example, the script will sample several demos as data to train. If I want to control the sample data, for example choosing a specified location to place the red ball to reach-target. Is there a convinient way to sample the demos I want?

You can override tasks and add/edit behaviour. For example, in your case, you could do something like this:

```
class ReachTarget(Task):

    def init_episode(self, index: int) -> List[str]:
         desc = super(ReachTarget, self).init_episode(index)
         # explicit set the positions, e.g.
         self.target.set_pose(...)
         return desc

```

Then when you create the rlbench env, you can pass `static_positions=True`, which will disable automatic task placement.
Repository: stepjam/RLBench
Issue: Not able to use notebook or ipython for interactive sessions
Root cause: Resource Mishandling

Hey folks,

This is a really nice work! I'm trying to use the `embed` function in IPython for creating interactive sessions so that it's easier to probe and play with the environment. However, after the environment starts simulation, I'm not able to call embed function, as the following error will appear:

`Traceback (most recent call last):
  File "tmp.py", line 52, in <module>
    embed()
  File "/home/txiao/anaconda3/lib/python3.7/site-packages/IPython/terminal/embed.py", line 387, in embed
    frame.f_code.co_filename, frame.f_lineno), **kwargs)
  File "/home/txiao/anaconda3/lib/python3.7/site-packages/traitlets/config/configurable.py", line 412, in instance
    inst = cls(*args, **kwargs)
  File "/home/txiao/anaconda3/lib/python3.7/site-packages/IPython/terminal/embed.py", line 159, in __init__
    super(InteractiveShellEmbed,self).__init__(**kw)
  File "/home/txiao/anaconda3/lib/python3.7/site-packages/IPython/terminal/interactiveshell.py", line 460, in __init__
    self.init_prompt_toolkit_cli()
  File "/home/txiao/anaconda3/lib/python3.7/site-packages/IPython/terminal/interactiveshell.py", line 292, in init_prompt_toolkit_cli
    **self._extra_prompt_options())
  File "/home/txiao/anaconda3/lib/python3.7/site-packages/prompt_toolkit/shortcuts/prompt.py", line 344, in __init__
    output = output or get_default_output()
  File "/home/txiao/anaconda3/lib/python3.7/site-packages/prompt_toolkit/output/defaults.py", line 67, in get_default_output
    return create_output()
  File "/home/txiao/anaconda3/lib/python3.7/site-packages/prompt_toolkit/output/defaults.py", line 45, in create_output
    stdout, term=get_term_environment_variable())
  File "/home/txiao/anaconda3/lib/python3.7/site-packages/prompt_toolkit/output/vt100.py", line 420, in from_pty
    isatty = stdout.isatty()
ValueError: I/O operation on closed file`

I'm wondering whether it's a desirable behavior, or I'm simply not supposed to do so.

Thanks!

Hi,
This is probably a PyRep issue rather than RLBench. I know that PyRep has trouble with anaconda (which you seem to be using). Can you try what you are doing with PyRep directly and see if you get a similar issue. Also see these PyRep issues:

https://github.com/stepjam/PyRep/issues/32
https://github.com/stepjam/PyRep/issues/36

Hey,

Thanks for the quick reply! I've tried it on examples on PyRep, and it seems like the error is the same. 

I don't think this one is related to Anaconda. I tried using original environment and encountered the same error.

More specifically, I tried `example_turtlebot_navigation.py` in PyRep. As long as I insert `embed` after simulation starts, e.g., in the action loop, the program throws out the error. Even at the end of program I had this error.

Shall I keep this thread or raise an issue in PyRep?

Thanks.

Hi,
Sorry for the delay in getting back to you (I've been away).
Yes, please raise this in PyRep, I will address it there. Thanks for raising!

Best,
Stephen
Repository: stepjam/RLBench
Issue: A potential bug ?
Root cause: Documentation Deficiency

Hello, 

Is there a bug in this line (https://github.com/stepjam/RLBench/blob/master/rlbench/backend/task.py#L268)? The function seems to be returning success as both elements in the tuple. As far as I understand, the second element should be related to termination as expected here (https://github.com/stepjam/RLBench/blob/master/rlbench/task_environment.py#L289).

Also, are these lines unnecessary (https://github.com/stepjam/RLBench/blob/master/rlbench/task_environment.py#L88-L89)?

Best, Ankit


Hi. Thanks for pointing out :)
Not a bug; this was just a feature that I did not end up using (the second value in the tuple is never used anywhere). I plan to remove it in a refactor sweep at some point, but this wont change the behaviour of how success operates.

> Also, are these lines unnecessary (https://github.com/stepjam/RLBench/blob/master/rlbench/task_environment.py#L88-L89)?

It seems like they aren't necessary. Will take these out in the 1.0.9 release. I'll leave this open until I do these refactors.
Repository: stepjam/RLBench
Issue: task_bulider

where can I open the task builder? Should I  find it in the backend file in RLbench ?

It's located in the [tools/ directory](https://github.com/stepjam/RLBench/tree/master/tools).

I have new task building video tutorials coming out Monday or Tuesday to replace the old text tutorials, so hopefully they will be helpful for you.
Repository: stepjam/RLBench
Issue: Import Ur5 Manipulator
Root cause: Documentation Deficiency

How can I use ur5 arm in Rlbench?

Please see this: https://github.com/stepjam/RLBench/tree/1.0.8#swapping-arms
Ur5 is in PyRep, and so we can bring this into RLBench. I should have time to get this done today or tomorrow.

Awesome, Thank You
Repository: stepjam/RLBench
Issue: Using robotic arms other than "panda" will report errors
Root cause: Validation Gap

When creating an environment, read robot_configuration ='jaco' or'mico' or'sawyer' and an error will be reported: rlbench.task_environment.TaskEnvironmentError: Could not place the task reach_target in the scene. This should not happen, please raise an issues on this task .And Coppeliasim reported an error:[simExtIK:error]Found invalid joint handle(s).

Hi. What task do you get this error with?

> Hi. What task do you get this error with?
Many tasks went wrong, such as ReachTarget.
An error was reported when calling ‘python imitation_learning.py’, I only changed the parameters:
env = Environment(action_mode,DATASET,obs_config,False,**robot_configuration='jaco'**).
thanks~

Hi @fanSunny , I have the same problem with you, have you try successfully in 'sawyer'?

> Hi @fanSunny , I have the same problem with you, have you try successfully in 'sawyer'?

Hi@tunglm2203,I tried using ‘sawyer’ but it was still unsuccessful.

I assume this is related to #94 .
Fix will be here by end of week
Repository: stepjam/RLBench
Issue: Questions about the arrangement of waypoint points

When looking at the task demonstration provided by rlbench, I found that the waypoint settings for object grabbing are somewhat different. Some objects have a waypoint set before close_gripper (for example, block grabbing in BlockPyramid). Some objects have no settings, and the robotic arm moves directly to the object grabbing. Take the waypoint.(For example, catch plate in SetTheTable).So what is the rule for setting waypoint?

Hi
That is up to the task designer :) 
When building my tasks, I prefer to place a 'pre-grasp' waypoint above objects before the actual grasp waypoint, but that not always necessary. Hope that helps.

@stepjam Thanks for your amazing work. I have one question about waypoints.
If I want the robot gripper to touch a specific part of the object (e.g., left border of the object), is there any way instead of using the waypoint? Using only the waypoint did not help. I am now referring to this video (https://www.youtube.com/watch?v=bKaK_9O3v7Y).

p.s. If you need more explanations, please feel free to ask.
Repository: stepjam/RLBench
Issue: [Bug]Creating Scene via Task Builder 
Root cause: Validation Gap

Hi, 
I installed PyRep and RLBenchmark. They together seem promising. I have encountered two bugs while following the "Simple Task Walkthrough". 

1)
When trying to run the task builder, it was not able to find the task validator. So I needed to change the import at the start of task builder 
from
from tools.task_validator import task_smoke, TaskValidationError
to 
from task_validator import task_smoke, TaskValidationError

2) 
While trying to create a non-existing task, at first it says that it does not exist in the system, would you like to create it. However, when you want to create it the system throws errors as follows: (XXXX is the new task name)
ModuleNotFoundError: No module named 'rlbench.tasks.XXXX'
rlbench.utils.InvalidTaskName: The task file 'XXXX' does not exist or cannot be compiled.

And a small note, the task in the tutorial slide_block_to_target already exists, you may want to change it as in the screenshot to slide_block

Best




I found the second BUG too.
But I still haven't found solution.
Do u solved this?

Ran into similar but reinstall & re-setup fixed it